sc-1 questions

profilejeekay89
Numerical_Methods_for_Engineers_7th_Edit.pdf

Numerical Methods for Engineers

SEVENTH EDITION

Steven C. Chapra Berger Chair in Computing and Engineering

Tufts University

Raymond P. Canale Professor Emeritus of Civil Engineering

University of Michigan

cha9792x_fm_i_xvi.indd Page i 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page i 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

NUMERICAL METHODS FOR ENGINEERS, SEVENTH EDITION

Published by McGraw-Hill Education, 2 Penn Plaza, New York, NY 10121. Copyright © 2015 by McGraw-Hill Education. All rights reserved. Printed in the United States of America. Previous editions © 2010, 2006, and 2002. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written consent of McGraw-Hill Education, including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning.

Some ancillaries, including electronic and print components, may not be available to customers outside the United States.

This book is printed on acid-free paper.

1 2 3 4 5 6 7 8 9 0 DOC/DOC 1 0 9 8 7 6 5 4

ISBN 978–0–07–339792–4 MHID 0–07–339792–x

Senior Vice President, Products & Markets: Kurt L. Strand Vice President, General Manager, Products & Markets: Marty Lange Vice President, Content Production & Technology Services: Kimberly Meriwether David Executive Brand Manager: Bill Stenquist Managing Director: Thomas Timp Global Publisher: Raghothaman Srinivasan Developmental Editor: Lorraine Buczek Marketing Manager: Heather Wagner Director, Content Production: Terri Schiesl Senior Content Project Manager: Melissa M. Leick Buyer: Jennifer Pickel Cover Designer: Studio Montage, St. Louis, MO Cover Image: Peak towering above clouds: Royalty-Free/CORBIS; Skysurfers: Getty Images/Digital Vision/RF Media Project Manager: Sandra M. Schnee Compositor: Aptara®, Inc. Typeface: 10/12 Time Roman Printer: R. R. Donnelley

All credits appearing on page or at the end of the book are considered to be an extension of the copyright page.

Library of Congress Cataloging-in-Publication Data

Chapra, Steven C. Numerical methods for engineers / Steven C. Chapra, Berger chair in computing and engineering, Tufts University, Raymond P. Canale, professor emeritus of civil engineering, University of Michigan. — Seventh edition. pages cm Includes bibliographical references and index. ISBN 978-0-07-339792-4 (alk. paper) — ISBN 0-07-339792-X (alk. paper) 1. Engineering mathematics—Data processing. 2. Numerical calculations—Data processing 3. Microcomputers—Programming. I. Canale, Raymond P. II. Title. TA345.C47 2015 518.024’62—dc23 2013041704

cha9792x_fm_i_xvi.indd Page ii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page ii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

To

Margaret and Gabriel Chapra

Helen and Chester Canale

cha9792x_fm_i_xvi.indd Page iii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page iii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

iv

CONTENTS

PREFACE xiv

ABOUT THE AUTHORS xvi

PART ONE

MODELING, PT1.1 Motivation 3 COMPUTERS, AND PT1.2 Mathematical Background 5 ERROR ANALYSIS 3 PT1.3 Orientation 8

CHAPTER 1 Mathematical Modeling and Engineering Problem Solving 11

1.1 A Simple Mathematical Model 11 1.2 Conservation Laws and Engineering 18 Problems 21

CHAPTER 2 Programming and Software 27

2.1 Packages and Programming 27 2.2 Structured Programming 28 2.3 Modular Programming 37 2.4 Excel 39 2.5 MATLAB 43 2.6 Mathcad 47 2.7 Other Languages and Libraries 48 Problems 49

CHAPTER 3 Approximations and Round-Off Errors 55

3.1 Signifi cant Figures 56 3.2 Accuracy and Precision 58 3.3 Error Defi nitions 59 3.4 Round-Off Errors 65 Problems 79

cha9792x_fm_i_xvi.indd Page iv 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page iv 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

CONTENTS v

CHAPTER 4 Truncation Errors and the Taylor Series 81

4.1 The Taylor Series 81 4.2 Error Propagation 97 4.3 Total Numerical Error 101 4.4 Blunders, Formulation Errors, and Data Uncertainty 106 Problems 108

EPILOGUE: PART ONE 110 PT1.4 Trade-Offs 110 PT1.5 Important Relationships and Formulas 113 PT1.6 Advanced Methods and Additional References 113

PART TWO

ROOTS OF PT2.1 Motivation 117 EQUATIONS 117 PT2.2 Mathematical Background 119

PT2.3 Orientation 120

CHAPTER 5 Bracketing Methods 123

5.1 Graphical Methods 123 5.2 The Bisection Method 127 5.3 The False-Position Method 135 5.4 Incremental Searches and Determining Initial Guesses 141 Problems 142

CHAPTER 6 Open Methods 145

6.1 Simple Fixed-Point Iteration 146 6.2 The Newton-Raphson Method 151 6.3 The Secant Method 157 6.4 Brent’s Method 162 6.5 Multiple Roots 166 6.6 Systems of Nonlinear Equations 169 Problems 173

CHAPTER 7 Roots of Polynomials 176

7.1 Polynomials in Engineering and Science 176 7.2 Computing with Polynomials 179 7.3 Conventional Methods 182

cha9792x_fm_i_xvi.indd Page v 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page v 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

vi CONTENTS

7.4 Müller’s Method 183 7.5 Bairstow’s Method 187 7.6 Other Methods 192 7.7 Root Location with Software Packages 192 Problems 202

CHAPTER 8 Case Studies: Roots of Equations 204

8.1 Ideal and Nonideal Gas Laws (Chemical/Bio Engineering) 204 8.2 Greenhouse Gases and Rainwater (Civil/Environmental Engineering) 207 8.3 Design of an Electric Circuit (Electrical Engineering) 209 8.4 Pipe Friction (Mechanical/Aerospace Engineering) 212 Problems 215

EPILOGUE: PART TWO 226 PT2.4 Trade-Offs 226 PT2.5 Important Relationships and Formulas 227 PT2.6 Advanced Methods and Additional References 227

PART THREE

LINEAR ALGEBRAIC PT3.1 Motivation 231 EQUATIONS 231 PT3.2 Mathematical Background 233

PT3.3 Orientation 241

CHAPTER 9 Gauss Elimination 245

9.1 Solving Small Numbers of Equations 245 9.2 Naive Gauss Elimination 252 9.3 Pitfalls of Elimination Methods 258 9.4 Techniques for Improving Solutions 264 9.5 Complex Systems 271 9.6 Nonlinear Systems of Equations 271 9.7 Gauss-Jordan 273 9.8 Summary 275 Problems 275

CHAPTER 10 LU Decomposition and Matrix Inversion 278 10.1 LU Decomposition 278 10.2 The Matrix Inverse 287 10.3 Error Analysis and System Condition 291 Problems 297

cha9792x_fm_i_xvi.indd Page vi 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page vi 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

CONTENTS vii

CHAPTER 11 Special Matrices and Gauss-Seidel 300

11.1 Special Matrices 300 11.2 Gauss-Seidel 304 11.3 Linear Algebraic Equations with Software Packages 311 Problems 316

CHAPTER 12 Case Studies: Linear Algebraic Equations 319

12.1 Steady-State Analysis of a System of Reactors (Chemical/Bio Engineering) 319 12.2 Analysis of a Statically Determinate Truss (Civil/Environmental Engineering) 322 12.3 Currents and Voltages in Resistor Circuits (Electrical Engineering) 326 12.4 Spring-Mass Systems (Mechanical/Aerospace Engineering) 328 Problems 331

EPILOGUE: PART THREE 341 PT3.4 Trade-Offs 341 PT3.5 Important Relationships and Formulas 342 PT3.6 Advanced Methods and Additional References 342

PART FOUR

OPTIMIZATION 345 PT4.1 Motivation 345 PT4.2 Mathematical Background 350 PT4.3 Orientation 351

CHAPTER 13 One-Dimensional Unconstrained Optimization 355

13.1 Golden-Section Search 356 13.2 Parabolic Interpolation 363 13.3 Newton’s Method 365 13.4 Brent’s Method 366 Problems 368

CHAPTER 14 Multidimensional Unconstrained Optimization 370

14.1 Direct Methods 371 14.2 Gradient Methods 375 Problems 388

cha9792x_fm_i_xvi.indd Page vii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page vii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

viii CONTENTS

CHAPTER 15 Constrained Optimization 390

15.1 Linear Programming 390 15.2 Nonlinear Constrained Optimization 401 15.3 Optimization with Software Packages 402 Problems 413

CHAPTER 16 Case Studies: Optimization 416

16.1 Least-Cost Design of a Tank (Chemical/Bio Engineering) 416 16.2 Least-Cost Treatment of Wastewater (Civil/Environmental Engineering) 421 16.3 Maximum Power Transfer for a Circuit (Electrical Engineering) 425 16.4 Equilibrium and Minimum Potential Energy (Mechanical/Aerospace Engineering) 429 Problems 431

EPILOGUE: PART FOUR 438 PT4.4 Trade-Offs 438 PT4.5 Additional References 439

PART FIVE

CURVE FITTING 441 PT5.1 Motivation 441 PT5.2 Mathematical Background 443 PT5.3 Orientation 452

CHAPTER 17 Least-Squares Regression 456

17.1 Linear Regression 456 17.2 Polynomial Regression 472 17.3 Multiple Linear Regression 476 17.4 General Linear Least Squares 479 17.5 Nonlinear Regression 483 Problems 487

CHAPTER 18 Interpolation 490

18.1 Newton’s Divided-Difference Interpolating Polynomials 491 18.2 Lagrange Interpolating Polynomials 502 18.3 Coeffi cients of an Interpolating Polynomial 507 18.4 Inverse Interpolation 507 18.5 Additional Comments 508 18.6 Spline Interpolation 511 18.7 Multidimensional Interpolation 521 Problems 524

cha9792x_fm_i_xvi.indd Page viii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page viii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

CONTENTS ix

CHAPTER 19 Fourier Approximation 526

19.1 Curve Fitting with Sinusoidal Functions 527 19.2 Continuous Fourier Series 533 19.3 Frequency and Time Domains 536 19.4 Fourier Integral and Transform 540 19.5 Discrete Fourier Transform (DFT) 542 19.6 Fast Fourier Transform (FFT) 544 19.7 The Power Spectrum 551 19.8 Curve Fitting with Software Packages 552 Problems 561

CHAPTER 20 Case Studies: Curve Fitting 563

20.1 Linear Regression and Population Models (Chemical/Bio Engineering) 563 20.2 Use of Splines to Estimate Heat Transfer (Civil/Environmental Engineering) 567 20.3 Fourier Analysis (Electrical Engineering) 569 20.4 Analysis of Experimental Data (Mechanical/Aerospace Engineering) 570 Problems 572

EPILOGUE: PART FIVE 582 PT5.4 Trade-Offs 582 PT5.5 Important Relationships and Formulas 583 PT5.6 Advanced Methods and Additional References 584

PART SIX

NUMERICAL PT6.1 Motivation 587 DIFFERENTIATION PT6.2 Mathematical Background 597 AND PT6.3 Orientation 599 INTEGRATION 587

CHAPTER 21 Newton-Cotes Integration Formulas 603

21.1 The Trapezoidal Rule 605 21.2 Simpson’s Rules 615 21.3 Integration with Unequal Segments 624 21.4 Open Integration Formulas 627 21.5 Multiple Integrals 627 Problems 629

cha9792x_fm_i_xvi.indd Page ix 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page ix 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

x CONTENTS

CHAPTER 22 Integration of Equations 633

22.1 Newton-Cotes Algorithms for Equations 633 22.2 Romberg Integration 634 22.3 Adaptive Quadrature 640 22.4 Gauss Quadrature 642 22.5 Improper Integrals 650 Problems 653

CHAPTER 23 Numerical Differentiation 655

23.1 High-Accuracy Differentiation Formulas 655 23.2 Richardson Extrapolation 658 23.3 Derivatives of Unequally Spaced Data 660 23.4 Derivatives and Integrals for Data with Errors 661 23.5 Partial Derivatives 662 23.6 Numerical Integration/Differentiation with Software Packages 663 Problems 670

CHAPTER 24 Case Studies: Numerical Integration and Differentiation 673

24.1 Integration to Determine the Total Quantity of Heat (Chemical/Bio Engineering) 673

24.2 Effective Force on the Mast of a Racing Sailboat (Civil/Environmental Engineering) 675

24.3 Root-Mean-Square Current by Numerical Integration (Electrical Engineering) 677

24.4 Numerical Integration to Compute Work (Mechanical/Aerospace Engineering) 680

Problems 684

EPILOGUE: PART SIX 694 PT6.4 Trade-Offs 694 PT6.5 Important Relationships and Formulas 695 PT6.6 Advanced Methods and Additional References 695

PART SEVEN

ORDINARY PT7.1 Motivation 699 DIFFERENTIAL PT7.2 Mathematical Background 703 EQUATIONS 699 PT7.3 Orientation 705

cha9792x_fm_i_xvi.indd Page x 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page x 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

CONTENTS xi

CHAPTER 25 Runge-Kutta Methods 709

25.1 Euler’s Method 710 25.2 Improvements of Euler’s Method 721 25.3 Runge-Kutta Methods 729 25.4 Systems of Equations 739 25.5 Adaptive Runge-Kutta Methods 744 Problems 752

CHAPTER 26 Stiffness and Multistep Methods 755

26.1 Stiffness 755 26.2 Multistep Methods 759 Problems 779

CHAPTER 27 Boundary-Value and Eigenvalue Problems 781

27.1 General Methods for Boundary-Value Problems 782 27.2 Eigenvalue Problems 789 27.3 Odes and Eigenvalues with Software Packages 801 Problems 808

CHAPTER 28 Case Studies: Ordinary Differential Equations 811

28.1 Using ODEs to Analyze the Transient Response of a Reactor (Chemical/Bio Engineering) 811

28.2 Predator-Prey Models and Chaos (Civil/Environmental Engineering) 818 28.3 Simulating Transient Current for an Electric Circuit (Electrical Engineering) 822 28.4 The Swinging Pendulum (Mechanical/Aerospace Engineering) 827 Problems 831

EPILOGUE: PART SEVEN 841 PT7.4 Trade-Offs 841 PT7.5 Important Relationships and Formulas 842 PT7.6 Advanced Methods and Additional References 842

PART EIGHT

PARTIAL PT8.1 Motivation 845 DIFFERENTIAL PT8.2 Orientation 848 EQUATIONS 845

cha9792x_fm_i_xvi.indd Page xi 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xi 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

xii CONTENTS

CHAPTER 29 Finite Difference: Elliptic Equations 852

29.1 The Laplace Equation 852 29.2 Solution Technique 854 29.3 Boundary Conditions 860 29.4 The Control-Volume Approach 866 29.5 Software to Solve Elliptic Equations 869 Problems 870

CHAPTER 30 Finite Difference: Parabolic Equations 873

30.1 The Heat-Conduction Equation 873 30.2 Explicit Methods 874 30.3 A Simple Implicit Method 878 30.4 The Crank-Nicolson Method 882 30.5 Parabolic Equations in Two Spatial Dimensions 885 Problems 888

CHAPTER 31 Finite-Element Method 890

31.1 The General Approach 891 31.2 Finite-Element Application in One Dimension 895 31.3 Two-Dimensional Problems 904 31.4 Solving PDEs with Software Packages 908 Problems 912

CHAPTER 32 Case Studies: Partial Differential Equations 915

32.1 One-Dimensional Mass Balance of a Reactor (Chemical/Bio Engineering) 915

32.2 Defl ections of a Plate (Civil/Environmental Engineering) 919 32.3 Two-Dimensional Electrostatic Field Problems (Electrical

Engineering) 921 32.4 Finite-Element Solution of a Series of Springs

(Mechanical/Aerospace Engineering) 924 Problems 928

EPILOGUE: PART EIGHT 931 PT8.3 Trade-Offs 931 PT8.4 Important Relationships and Formulas 931 PT8.5 Advanced Methods and Additional References 932

cha9792x_fm_i_xvi.indd Page xii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

CONTENTS xiii

APPENDIX A: THE FOURIER SERIES 933

APPENDIX B: GETTING STARTED WITH MATLAB 935

APPENDIX C: GETTING STARTED WITH MATHCAD 943

BIBLIOGRAPHY 954

INDEX 957

cha9792x_fm_i_xvi.indd Page xiii 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xiii 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

xiv

PREFACE

It has been over twenty years since we published the fi rst edition of this book. Over that period, our original contention that numerical methods and computers would fi gure more prominently in the engineering curriculum—particularly in the early parts—has been dra- matically borne out. Many universities now offer freshman, sophomore, and junior courses in both introductory computing and numerical methods. In addition, many of our colleagues are integrating computer-oriented problems into other courses at all levels of the curriculum. Thus, this new edition is still founded on the basic premise that student engineers should be provided with a strong and early introduction to numerical methods. Consequently, although we have expanded our coverage in the new edition, we have tried to maintain many of the features that made the fi rst edition accessible to both lower- and upper-level undergraduates. These include:

• Problem Orientation. Engineering students learn best when they are motivated by problems. This is particularly true for mathematics and computing. Consequently, we have approached numerical methods from a problem-solving perspective.

• Student-Oriented Pedagogy. We have developed a number of features to make this book as student-friendly as possible. These include the overall organization, the use of introductions and epilogues to consolidate major topics and the extensive use of worked examples and case studies from all areas of engineering. We have also en- deavored to keep our explanations straightforward and oriented practically.

• Computational Tools. We empower our students by helping them utilize the standard “point-and-shoot” numerical problem-solving capabilities of packages like Excel, MATLAB, and Mathcad software. However, students are also shown how to develop simple, well-structured programs to extend the base capabilities of those environ- ments. This knowledge carries over to standard programming languages such as Visual Basic, Fortran 90, and C/C11. We believe that the current fl ight from computer programming represents something of a “dumbing down” of the engineering curricu- lum. The bottom line is that as long as engineers are not content to be tool limited, they will have to write code. Only now they may be called “macros” or “M-fi les.” This book is designed to empower them to do that.

Beyond these fi ve original principles, the seventh edition has new and expanded problem sets. Most of the problems have been modifi ed so that they yield different numerical solu- tions from previous editions. In addition, a variety of new problems have been included. The seventh edition also includes McGraw-Hill’s Connect® Engineering. This online homework management tool allows assignment of algorithmic problems for homework, quizzes, and tests. It connects students with the tools and resources they’ll need to achieve success. To learn more, visit www.mcgrawhillconnect.com. McGraw-Hill LearnSmart™ is also available as an integrated feature of McGraw-Hill Connect® Engineering. It is an adaptive learning system designed to help students learn faster, study more effi ciently, and retain more knowledge for greater success. LearnSmart assesses

cha9792x_fm_i_xvi.indd Page xiv 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xiv 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PREFACE xv

a student’s knowledge of course content through a series of adaptive questions. It pinpoints concepts the student does not understand and maps out a personalized study plan for success. Visit the following site for a demonstration. www.mhlearnsmart.com As always, our primary intent in writing this book is to provide students with a sound introduction to numerical methods. We believe that motivated students who enjoy numeri- cal methods, computers, and mathematics will, in the end, make better engineers. If our book fosters an enthusiasm for these subjects, we will consider our efforts a success.

Acknowledgments. We would like to thank our friends at McGraw-Hill. In particular, Lorraine Buczek and Bill Stenquist, who provided a positive and supportive atmosphere for creating this edition. As usual, Beatrice Sussman did a masterful job of copyediting the man- uscript and Arpana Kumari of Aptara also did an outstanding job in the book’s fi nal production phase. As in past editions, David Clough (University of Colorado), Mike Gustafson (Duke), and Jerry Stedinger (Cornell University) generously shared their insights and suggestions. Use- ful suggestions were also made by Bill Philpot (Cornell University), Jim Guilkey (University of Utah), Dong-Il Seo (Chungnam National University, Korea), Niall Broekhuizen (NIWA, New Zealand), and Raymundo Cordero and Karim Muci (ITESM, Mexico). The present edition has also benefi ted from the reviews and suggestions by the following colleagues:

Betty Barr, University of Houston Jalal Behzadi, Shahid Chamran University Jordan Berg, Texas Tech University Jacob Bishop, Utah State University Estelle M. Eke, California State University, Sacramento Yazan A. Hussain, Jordan University of Science & Technology Yogesh Jaluria, Rutgers University S. Graham Kelly, The University of Akron Subha Kumpaty, Milwaukee School of Engineering Eckart Meiburg, University of California-Santa Barbara Prashant Mhaskar, McMaster University Luke Olson, University of Illinois at Urbana-Champaign Richard Pates Jr., Old Dominion University Joseph H. Pierluissi, University of Texas at El Paso Juan Perán, Universidad Nacional de Educación a Distancia (UNED) Scott A. Socolofsky, Texas A&M University

It should be stressed that although we received useful advice from the aforementioned individuals, we are responsible for any inaccuracies or mistakes you may detect in this edi- tion. Please contact Steve Chapra via e-mail if you should detect any errors in this edition. Finally, we would like to thank our family, friends, and students for their enduring patience and support. In particular, Cynthia Chapra, Danielle Husley, and Claire Canale are always there providing understanding, perspective, and love.

Steven C. Chapra Medford, Massachusetts [email protected]

Raymond P. Canale Lake Leelanau, Michigan

cha9792x_fm_i_xvi.indd Page xv 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xv 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

xvi

ABOUT THE AUTHORS

Steve Chapra teaches in the Civil and Environmental Engineering Department at Tufts University where he holds the Louis Berger Chair in Computing and Engineering. His other books include Surface Water-Quality Modeling and Applied Numerical Methods with MATLAB. Dr. Chapra received engineering degrees from Manhattan College and the University of Michigan. Before joining the faculty at Tufts, he worked for the Environmental Pro- tection Agency and the National Oceanic and Atmospheric Administration, and taught at Texas A&M University and the University of Colorado. His general research interests focus on surface water-quality modeling and advanced computer applications in environ- mental engineering. He is a Fellow of the ASCE, and has received a number of awards for his scholarly contributions, including the Rudolph Hering Medal (ASCE), and the Meriam-Wiley Distinguished Author Award (American Society for Engineering Education). He has also been recognized as the outstanding teacher among the engineering faculties at Texas A&M University, the University of Colorado, and Tufts University. Raymond P. Canale is an emeritus professor at the University of Michigan. During his over 20-year career at the university, he taught numerous courses in the area of comput- ers, numerical methods, and environmental engineering. He also directed extensive research programs in the area of mathematical and computer modeling of aquatic ecosystems. He has authored or coauthored several books and has published over 100 scientifi c papers and reports. He has also designed and developed personal computer software to facilitate en- gineering education and the solution of engineering problems. He has been given the Meriam-Wiley Distinguished Author Award by the American Society for Engineering Education for his books and software and several awards for his technical publications. Professor Canale is now devoting his energies to applied problems, where he works with engineering fi rms and industry and governmental agencies as a consultant and expert witness.

cha9792x_fm_i_xvi.indd Page xvi 22/11/13 6:42 PM F-468 cha9792x_fm_i_xvi.indd Page xvi 22/11/13 6:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

Numerical Methods for Engineers

cha9792x_p01_001-010.indd Page 1 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 1 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART ONE

cha9792x_p01_001-010.indd Page 2 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 2 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3

PT1.1 MOTIVATION

Numerical methods are techniques by which mathematical problems are formulated so that they can be solved with arithmetic operations. Although there are many kinds of numerical methods, they have one common characteristic: they invariably involve large numbers of tedious arithmetic calculations. It is little wonder that with the development of fast, effi cient digital computers, the role of numerical methods in engineering problem solving has increased dramatically in recent years.

PT1.1.1 Noncomputer Methods

Beyond providing increased computational fi repower, the widespread availability of com- puters (especially personal computers) and their partnership with numerical methods has had a signifi cant infl uence on the actual engineering problem-solving process. In the precomputer era there were generally three different ways in which engineers approached problem solving:

1. Solutions were derived for some problems using analytical, or exact, methods. These solutions were often useful and provided excellent insight into the behavior of some systems. However, analytical solutions can be derived for only a limited class of problems. These include those that can be approximated with linear models and those that have simple geometry and low dimensionality. Consequently, analytical solutions are of limited practical value because most real problems are nonlinear and involve complex shapes and processes.

2. Graphical solutions were used to characterize the behavior of systems. These graphical solutions usually took the form of plots or nomographs. Although graphical techniques can often be used to solve complex problems, the results are not very precise. Furthermore, graphical solutions (without the aid of computers) are extremely tedious and awkward to implement. Finally, graphical techniques are often limited to problems that can be described using three or fewer dimensions.

3. Calculators and slide rules were used to implement numerical methods manually. Although in theory such approaches should be perfectly adequate for solving complex problems, in actuality several diffi culties are encountered. Manual calculations are slow and tedious. Furthermore, consistent results are elusive because of simple blunders that arise when numerous manual tasks are performed.

During the precomputer era, signifi cant amounts of energy were expended on the solution technique itself, rather than on problem defi nition and interpretation (Fig. PT1.1a). This unfortunate situation existed because so much time and drudgery were required to obtain numerical answers using precomputer techniques.

MODELING, COMPUTERS, AND ERROR ANALYSIS

cha9792x_p01_001-010.indd Page 3 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 3 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4 MODELING, COMPUTERS, AND ERROR ANALYSIS

Today, computers and numerical methods provide an alternative for such compli- cated calculations. Using computer power to obtain solutions directly, you can approach these calculations without recourse to simplifying assumptions or time-intensive tech- niques. Although analytical solutions are still extremely valuable both for problem solving and for providing insight, numerical methods represent alternatives that greatly enlarge your capabilities to confront and solve problems. As a result, more time is available for the use of your creative skills. Thus, more emphasis can be placed on problem formulation and solution interpretation and the incorporation of total system, or “holistic,” awareness (Fig. PT1.1b).

PT1.1.2 Numerical Methods and Engineering Practice

Since the late 1940s the widespread availability of digital computers has led to a veri- table explosion in the use and development of numerical methods. At fi rst, this growth was somewhat limited by the cost of access to large mainframe computers, and, conse- quently, many engineers continued to use simple analytical approaches in a signifi cant portion of their work. Needless to say, the recent evolution of inexpensive personal

FIGURE PT1.1 The three phases of engineering problem solving in (a) the precomputer and (b) the computer era. The sizes of the boxes indicate the level of emphasis directed toward each phase. Computers facilitate the implementation of solution techniques and thus allow more emphasis to be placed on the creative aspects of problem formulation and interpretation of results.

INTERPRETATION

Ease of calculation allows holistic thoughts and intuition to develop;

system sensitivity and behavior can be studied

FORMULATION

In-depth exposition of relationship of

problem to fundamental laws

SOLUTION

Easy-to-use computer method

(b)

INTERPRETATION

In-depth analysis limited by time-

consuming solution

FORMULATION

Fundamental laws explained

briefly

SOLUTION

Elaborate and often complicated method to make problem tractable

(a)

cha9792x_p01_001-010.indd Page 4 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 4 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT1.2 MATHEMATICAL BACKGROUND 5

computers has given us ready access to powerful computational capabilities. There are several additional reasons why you should study numerical methods:

1. Numerical methods are extremely powerful problem-solving tools. They are capable of handling large systems of equations, nonlinearities, and complicated geometries that are not uncommon in engineering practice and that are often impossible to solve analytically. As such, they greatly enhance your problem-solving skills.

2. During your careers, you may often have occasion to use commercially available prepackaged, or “canned,” computer programs that involve numerical methods. The intelligent use of these programs is often predicated on knowledge of the basic theory underlying the methods.

3. Many problems cannot be approached using canned programs. If you are conversant with numerical methods and are adept at computer programming, you can design your own programs to solve problems without having to buy or commission expensive software.

4. Numerical methods are an effi cient vehicle for learning to use computers. It is well known that an effective way to learn programming is to actually write computer programs. Because numerical methods are for the most part designed for implementation on computers, they are ideal for this purpose. Further, they are especially well-suited to illustrate the power and the limitations of computers. When you successfully implement numerical methods on a computer and then apply them to solve otherwise intractable problems, you will be provided with a dramatic demonstration of how computers can serve your professional development. At the same time, you will also learn to acknowledge and control the errors of approximation that are part and parcel of large-scale numerical calculations.

5. Numerical methods provide a vehicle for you to reinforce your understanding of mathematics. Because one function of numerical methods is to reduce higher mathematics to basic arithmetic operations, they get at the “nuts and bolts” of some otherwise obscure topics. Enhanced understanding and insight can result from this alternative perspective.

PT1.2 MATHEMATICAL BACKGROUND

Every part in this book requires some mathematical background. Consequently, the in- troductory material for each part includes a section, such as the one you are reading, on mathematical background. Because Part One itself is devoted to background material on mathematics and computers, this section does not involve a review of a specifi c math- ematical topic. Rather, we take this opportunity to introduce you to the types of math- ematical subject areas covered in this book. As summarized in Fig. PT1.2, these are

1. Roots of Equations (Fig. PT1.2a). These problems are concerned with the value of a variable or a parameter that satisfi es a single nonlinear equation. These problems are especially valuable in engineering design contexts where it is often impossible to explicitly solve design equations for parameters.

2. Systems of Linear Algebraic Equations (Fig. PT1.2b). These problems are similar in spirit to roots of equations in the sense that they are concerned with values that

cha9792x_p01_001-010.indd Page 5 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 5 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6 MODELING, COMPUTERS, AND ERROR ANALYSIS

f(x)

x

Root

x2

x1

Solution

Minimum

f(x)

x

Interpolation

f(x)

x

f(x)

x

Regression

f(x)

I

(a) Part 2: Roots of equations Solve f(x) = 0 for x.

(c) Part 4: Optimization

(b) Part 3: Linear algebraic equations Given the a’s and the c’s, solve a11x1 + a12x2 = c1 a21x1 + a22x2 = c2 for the x’s.

Determine x that gives optimum f(x).

(e) Part 6: Integration I = �ab f (x) dx Find the area under the curve.

(d) Part 5: Curve fitting

x

FIGURE PT1.2 Summary of the numerical methods covered in this book.

cha9792x_p01_001-010.indd Page 6 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 6 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT1.2 MATHEMATICAL BACKGROUND 7

satisfy equations. However, in contrast to satisfying a single equation, a set of values is sought that simultaneously satisfi es a set of linear algebraic equations. Such equations arise in a variety of problem contexts and in all disciplines of engineering. In particular, they originate in the mathematical modeling of large systems of interconnected elements such as structures, electric circuits, and fl uid networks. However, they are also encountered in other areas of numerical methods such as curve fi tting and differential equations.

3. Optimization (Fig. PT1.2c). These problems involve determining a value or values of an independent variable that correspond to a “best” or optimal value of a function. Thus, as in Fig. PT1.2c, optimization involves identifying maxima and minima. Such problems occur routinely in engineering design contexts. They also arise in a number of other numerical methods. We address both single- and multi-variable unconstrained optimization. We also describe constrained optimization with particular emphasis on linear programming.

4. Curve Fitting (Fig. PT1.2d). You will often have occasion to fi t curves to data points. The techniques developed for this purpose can be divided into two general categories: regression and interpolation. Regression is employed where there is a signifi cant degree of error associated with the data. Experimental results are often of this kind. For these situations, the strategy is to derive a single curve that represents the general trend of the data without necessarily matching any individual points. In contrast, interpolation is used where the objective is to determine intermediate values between relatively error-free data points. Such is usually the case for tabulated information. For these situations, the strategy is to fi t a curve directly through the data points and use the curve to predict the intermediate values.

5. Integration (Fig. PT1.2e). As depicted, a physical interpretation of numerical integration is the determination of the area under a curve. Integration has many

y

x

(g) Part 8: Partial differential equations Given

solve for u as a function of x and y

= f (x, y)� 2u

�x2 �2u �y2

+

t

Slope = f(ti, yi)

y

�t

ti ti + 1

( f ) Part 7: Ordinary differential equations Given

solve for y as a function of t.

yi + 1 = yi + f (ti , yi ) �t

� = f (t, y) dy dt

�y �t

FIGURE PT1.2 (concluded)

cha9792x_p01_001-010.indd Page 7 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 7 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8 MODELING, COMPUTERS, AND ERROR ANALYSIS

applications in engineering practice, ranging from the determination of the centroids of oddly shaped objects to the calculation of total quantities based on sets of discrete measurements. In addition, numerical integration formulas play an important role in the solution of differential equations.

6. Ordinary Differential Equations (Fig. PT1.2f ). Ordinary differential equations are of great signifi cance in engineering practice. This is because many physical laws are couched in terms of the rate of change of a quantity rather than the magnitude of the quantity itself. Examples range from population-forecasting models (rate of change of population) to the acceleration of a falling body (rate of change of velocity). Two types of problems are addressed: initial-value and boundary-value problems. In addition, the computation of eigenvalues is covered.

7. Partial Differential Equations (Fig. PT1.2g). Partial differential equations are used to characterize engineering systems where the behavior of a physical quantity is couched in terms of its rate of change with respect to two or more independent variables. Examples include the steady-state distribution of temperature on a heated plate (two spatial dimensions) or the time-variable temperature of a heated rod (time and one spatial dimension). Two fundamentally different approaches are employed to solve partial differential equations numerically. In the present text, we will emphasize fi nite-difference methods that approximate the solution in a pointwise fashion (Fig. PT1.2g). However, we will also present an introduction to fi nite-element methods, which use a piecewise approach.

PT1.3 ORIENTATION

Some orientation might be helpful before proceeding with our introduction to nu- merical methods. The following is intended as an overview of the material in Part One. In addition, some objectives have been included to focus your efforts when studying the material.

PT1.3.1 Scope and Preview

Figure PT1.3 is a schematic representation of the material in Part One. We have designed this diagram to provide you with a global overview of this part of the book. We believe that a sense of the “big picture” is critical to developing insight into numerical methods. When reading a text, it is often possible to become lost in technical details. Whenever you feel that you are losing the big picture, refer back to Fig. PT1.3 to reorient yourself. Every part of this book includes a similar fi gure. Figure PT1.3 also serves as a brief preview of the material covered in Part One. Chapter 1 is designed to orient you to numerical methods and to provide motivation by demonstrating how these techniques can be used in the engineering modeling process. Chapter 2 is an introduction and review of computer-related aspects of numerical meth- ods and suggests the level of computer skills you should acquire to effi ciently apply succeeding information. Chapters 3 and 4 deal with the important topic of error analysis, which must be understood for the effective use of numerical methods. In addition, an epilogue is included that introduces the trade-offs that have such great signifi cance for the effective implementation of numerical methods.

cha9792x_p01_001-010.indd Page 8 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 8 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT1.3 ORIENTATION 9

FIGURE PT1.3 Schematic of the organization of the material in Part One: Modeling, Computers, and Error Analysis.

CHAPTER 1 Mathematical Modeling and Engineering

Problem Solving

PART 1 Modeling,

Computers, and

Error Analysis

CHAPTER 2 Programming and Software

CHAPTER 3 Approximations and Round-Off

Errors

CHAPTER 4 Truncation

Errors and the Taylor Series

EPILOGUE

2.7 Languages and

libraries

2.6 Mathcad

2.5 MATLAB

2.4 Excel

2.3 Modular

programming

2.2 Structured

programming

2.1 Packages and programming

PT 1.2 Mathematical background

PT 1.6 Advanced methods

PT 1.5 Important formulas

4.4 Miscellaneous

errors

4.3 Total numerical

error

4.2 Error

propagation

4.1 Taylor series

3.4 Round-off

errors

3.1 Significant

figures

3.3 Error

definitions

3.2 Accuracy and

precision

PT 1.4 Trade-offs

PT 1.3 Orientation

PT 1.1 Motivation

1.2 Conservation

laws

1.1 A simple

model

cha9792x_p01_001-010.indd Page 9 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 9 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10 MODELING, COMPUTERS, AND ERROR ANALYSIS

TABLE PT1.1 Specifi c study objectives for Part One.

1. Recognize the difference between analytical and numerical solutions. 2. Understand how conservation laws are employed to develop mathematical models of physical

systems. 3. Defi ne top-down and modular design. 4. Delineate the rules that underlie structured programming. 5. Be capable of composing structured and modular programs in a high-level computer language. 6. Know how to translate structured fl owcharts and pseudocode into code in a high-level language. 7. Start to familiarize yourself with any software packages that you will be using in conjunction with

this text. 8. Recognize the distinction between truncation and round-off errors. 9. Understand the concepts of signifi cant fi gures, accuracy, and precision. 10. Recognize the difference between true relative error et, approximate relative error ea, and

acceptable error es, and understand how ea and es are used to terminate an iterative computation. 11. Understand how numbers are represented in digital computers and how this representation induces

round-off error. In particular, know the difference between single and extended precision. 12. Recognize how computer arithmetic can introduce and amplify round-off errors in calculations. In

particular, appreciate the problem of subtractive cancellation. 13. Understand how the Taylor series and its remainder are employed to represent continuous functions. 14. Know the relationship between fi nite divided differences and derivatives. 15. Be able to analyze how errors are propagated through functional relationships. 16. Be familiar with the concepts of stability and condition. 17. Familiarize yourself with the trade-offs outlined in the Epilogue of Part One.

PT1.3.2 Goals and Objectives

Study Objectives. Upon completing Part One, you should be adequately prepared to embark on your studies of numerical methods. In general, you should have gained a fundamental understanding of the importance of computers and the role of approxima- tions and errors in the implementation and development of numerical methods. In addi- tion to these general goals, you should have mastered each of the specifi c study objectives listed in Table PT1.1.

Computer Objectives. Upon completing Part One, you should have mastered suffi cient computer skills to develop your own software for the numerical methods in this text. You should be able to develop well-structured and reliable computer programs on the basis of pseudocode, fl owcharts, or other forms of algorithms. You should have developed the capability to document your programs so that they may be effectively employed by users. Finally, in addition to your own programs, you may be using software packages along with this book. Packages like Excel, Mathcad, or The MathWorks, Inc. MATLAB® pro- gram are examples of such software. You should become familiar with these packages, so that you will be comfortable using them to solve numerical problems later in the text.

cha9792x_p01_001-010.indd Page 10 09/10/13 9:56 PM F-468 cha9792x_p01_001-010.indd Page 10 09/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

1

11

Mathematical Modeling and Engineering Problem Solving

Knowledge and understanding are prerequisites for the effective implementation of any tool. No matter how impressive your tool chest, you will be hard-pressed to repair a car if you do not understand how it works. This is particularly true when using computers to solve engineering problems. Although they have great potential utility, computers are practically useless without a fundamental understanding of how engineering systems work. This understanding is initially gained by empirical means—that is, by observation and experiment. However, while such empirically derived information is essential, it is only half the story. Over years and years of observation and experiment, engineers and scientists have noticed that certain aspects of their empirical studies occur repeatedly. Such general behavior can then be expressed as fundamental laws that essentially embody the cumulative wisdom of past experience. Thus, most engineering problem solving employs the two-pronged approach of empiricism and theoretical analysis (Fig. 1.1). It must be stressed that the two prongs are closely coupled. As new measurements are taken, the generalizations may be modifi ed or new ones developed. Similarly, the general- izations can have a strong infl uence on the experiments and observations. In particular, generalizations can serve as organizing principles that can be employed to synthesize ob- servations and experimental results into a coherent and comprehensive framework from which conclusions can be drawn. From an engineering problem-solving perspective, such a framework is most useful when it is expressed in the form of a mathematical model. The primary objective of this chapter is to introduce you to mathematical modeling and its role in engineering problem solving. We will also illustrate how numerical meth- ods fi gure in the process.

1.1 A SIMPLE MATHEMATICAL MODEL

A mathematical model can be broadly defi ned as a formulation or equation that expresses the essential features of a physical system or process in mathematical terms. In a very general sense, it can be represented as a functional relationship of the form

Dependent variable

5 f aindependent variables

, parameters, forcing

functions b (1.1)

C H A P T E R 1

cha9792x_ch01_011-026.indd Page 11 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 11 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

where the dependent variable is a characteristic that usually refl ects the behavior or state of the system; the independent variables are usually dimensions, such as time and space, along which the system’s behavior is being determined; the parameters are refl ective of the system’s properties or composition; and the forcing functions are external infl uences acting upon the system. The actual mathematical expression of Eq. (1.1) can range from a simple algebraic relationship to large complicated sets of differential equations. For example, on the basis of his observations, Newton formulated his second law of motion, which states that the time rate of change of momentum of a body is equal to the resultant force acting on it. The mathematical expression, or model, of the second law is the well- known equation

F 5 ma (1.2)

where F 5 net force acting on the body (N, or kg m/s2), m 5 mass of the object (kg), and a 5 its acceleration (m/s2).

Implementation

Numeric or graphic results

Mathematical model

Problem definition

THEORY DATA

Problem-solving tools: computers, statistics, numerical methods,

graphics, etc.

Societal interfaces: scheduling, optimization,

communication, public interaction,

etc.

FIGURE 1.1 The engineering problem- solving process.

cha9792x_ch01_011-026.indd Page 12 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 12 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

1.1 A SIMPLE MATHEMATICAL MODEL 13

The second law can be recast in the format of Eq. (1.1) by merely dividing both sides by m to give

a 5 F m

(1.3)

where a 5 the dependent variable refl ecting the system’s behavior, F 5 the forcing function, and m 5 a parameter representing a property of the system. Note that for this simple case there is no independent variable because we are not yet predicting how acceleration varies in time or space. Equation (1.3) has several characteristics that are typical of mathematical models of the physical world:

1. It describes a natural process or system in mathematical terms. 2. It represents an idealization and simplifi cation of reality. That is, the model ignores

negligible details of the natural process and focuses on its essential manifestations. Thus, the second law does not include the effects of relativity that are of minimal importance when applied to objects and forces that interact on or about the earth’s surface at velocities and on scales visible to humans.

3. Finally, it yields reproducible results and, consequently, can be used for predictive purposes. For example, if the force on an object and the mass of an object are known, Eq. (1.3) can be used to compute acceleration.

Because of its simple algebraic form, the solution of Eq. (1.2) can be obtained eas- ily. However, other mathematical models of physical phenomena may be much more complex, and either cannot be solved exactly or require more sophisticated mathematical techniques than simple algebra for their solution. To illustrate a more complex model of this kind, Newton’s second law can be used to determine the terminal velocity of a free- falling body near the earth’s surface. Our falling body will be a parachutist (Fig. 1.2). A model for this case can be derived by expressing the acceleration as the time rate of change of the velocity (dy�dt) and substituting it into Eq. (1.3) to yield

dy

dt 5

F m

(1.4)

where y is velocity (m/s) and t is time (s). Thus, the mass multiplied by the rate of change of the velocity is equal to the net force acting on the body. If the net force is positive, the object will accelerate. If it is negative, the object will decelerate. If the net force is zero, the object’s velocity will remain at a constant level. Next, we will express the net force in terms of measurable variables and parameters. For a body falling within the vicinity of the earth (Fig. 1.2), the net force is composed of two opposing forces: the downward pull of gravity FD and the upward force of air resistance FU:

F 5 FD 1 FU (1.5)

If the downward force is assigned a positive sign, the second law can be used to formu- late the force due to gravity, as

FD 5 mg (1.6)

where g 5 the gravitational constant, or the acceleration due to gravity, which is approxi- mately equal to 9.81 m/s2.

FU

FD

FIGURE 1.2 Schematic diagram of the forces acting on a falling parachutist. FD is the downward force due to gravity. FU is the upward force due to air resistance.

cha9792x_ch01_011-026.indd Page 13 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 13 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

Air resistance can be formulated in a variety of ways. A simple approach is to as- sume that it is linearly proportional to velocity1 and acts in an upward direction, as in

FU 5 2cy (1.7)

where c 5 a proportionality constant called the drag coeffi cient (kg/s). Thus, the greater the fall velocity, the greater the upward force due to air resistance. The parameter c accounts for properties of the falling object, such as shape or surface roughness, that affect air resistance. For the present case, c might be a function of the type of jumpsuit or the orientation used by the parachutist during free-fall. The net force is the difference between the downward and upward force. Therefore, Eqs. (1.4) through (1.7) can be combined to yield

dy

dt 5

mg 2 cy

m (1.8)

or simplifying the right side,

dy

dt 5 g 2

c m

y (1.9)

Equation (1.9) is a model that relates the acceleration of a falling object to the forces acting on it. It is a differential equation because it is written in terms of the differential rate of change (dy�dt) of the variable that we are interested in predicting. However, in contrast to the solution of Newton’s second law in Eq. (1.3), the exact solution of Eq.  (1.9) for the velocity of the falling parachutist cannot be obtained using simple algebraic manipulation. Rather, more advanced techniques, such as those of calculus, must be applied to obtain an exact or analytical solution. For example, if the parachutist is initially at rest (y 5 0 at t 5 0), calculus can be used to solve Eq. (1.9) for

y(t) 5 gm

c (1 2 e2 (cym)t) (1.10)

Note that Eq. (1.10) is cast in the general form of Eq. (1.1), where y(t) 5 the dependent variable, t 5 the independent variable, c and m 5 parameters, and g 5 the forcing function.

EXAMPLE 1.1 Analytical Solution to the Falling Parachutist Problem

Problem Statement. A parachutist of mass 68.1 kg jumps out of a stationary hot air balloon. Use Eq. (1.10) to compute velocity prior to opening the chute. The drag coeffi cient is equal to 12.5 kg/s.

Solution. Inserting the parameters into Eq. (1.10) yields

y(t) 5 9.81(68.1)

12.5 (1 2 e2 (12.5y68.1)t) 5 53.44 (1 2 e20.18355t)

which can be used to compute

1In fact, the relationship is actually nonlinear and might better be represented by a power relationship such as FU 5 2cy

2. We will explore how such nonlinearities affect the model in problems at the end of this chapter.

cha9792x_ch01_011-026.indd Page 14 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 14 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

1.1 A SIMPLE MATHEMATICAL MODEL 15

t, s v, m/s

0 0.00 2 16.42 4 27.80 6 35.68 8 41.14 10 44.92 12 47.54 ` 53.44

According to the model, the parachutist accelerates rapidly (Fig. 1.3). A velocity of 44.92 m/s is attained after 10 s. Note also that after a suffi ciently long time, a constant veloc- ity, called the terminal velocity, of 53.44 m/s is reached. This velocity is constant because, eventually, the force of gravity will be in balance with the air resistance. Thus, the net force is zero and acceleration has ceased.

Equation (1.10) is called an analytical, or exact, solution because it exactly satisfi es the original differential equation. Unfortunately, there are many mathematical models that cannot be solved exactly. In many of these cases, the only alternative is to develop a numerical solution that approximates the exact solution. As mentioned previously, numerical methods are those in which the mathematical problem is reformulated so it can be solved by arithmetic operations. This can be illustrated

FIGURE 1.3 The analytical solution to the falling parachutist problem as computed in Example 1.1. Velocity increases with time and asymptotically approaches a terminal velocity.

0 0

20

40

4 8 12

t, s

v, m

/s

Terminal velocity

cha9792x_ch01_011-026.indd Page 15 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 15 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

for Newton’s second law by realizing that the time rate of change of velocity can be approximated by (Fig. 1.4):

dy

dt > ¢y ¢t

5 y(ti11) 2 y(ti)

ti11 2 ti (1.11)

where Dy and Dt 5 differences in velocity and time, respectively, computed over fi nite intervals, y(ti) 5 velocity at an initial time ti, and y(ti+1) 5 velocity at some later time ti+1. Note that dy/dt > ¢yy¢t is approximate because Dt is fi nite. Remember from calculus that

dy

dt 5 lim ¢tS0

¢y ¢t

Equation (1.11) represents the reverse process. Equation (1.11) is called a fi nite divided difference approximation of the derivative at time ti. It can be substituted into Eq. (1.9) to give

y(ti11) 2 y(ti)

ti11 2 ti 5 g 2

c m

y(ti)

This equation can then be rearranged to yield

y(ti11) 5 y(ti) 1 cg 2 cm y(ti) d (ti11 2 ti) (1.12) Notice that the term in brackets is the right-hand side of the differential equation itself [Eq. (1.9)]. That is, it provides a means to compute the rate of change or slope of y. Thus, the differential equation has been transformed into an equation that can be used to determine the velocity algebraically at ti11 using the slope and previous values of

FIGURE 1.4 The use of a fi nite difference to approximate the fi rst derivative of v with respect to t.

v(ti +1)

v(ti )

�v

True slope dv/dt

Approximate slope �v �t

v(ti +1) – v(ti ) ti +1 – ti

=

ti +1ti t

�t

cha9792x_ch01_011-026.indd Page 16 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 16 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

1.1 A SIMPLE MATHEMATICAL MODEL 17

y and t. If you are given an initial value for velocity at some time ti, you can easily com- pute velocity at a later time ti11. This new value of velocity at ti11 can in turn be employed to extend the computation to velocity at ti12 and so on. Thus, at any time along the way,

New value 5 old value 1 slope 3 step size

Note that this approach is formally called Euler’s method.

EXAMPLE 1.2 Numerical Solution to the Falling Parachutist Problem

Problem Statement. Perform the same computation as in Example 1.1 but use Eq. (1.12) to compute the velocity. Employ a step size of 2 s for the calculation.

Solution. At the start of the computation (ti 5 0), the velocity of the parachutist is zero. Using this information and the parameter values from Example 1.1, Eq. (1.12) can be used to compute velocity at ti11 5 2 s:

y 5 0 1 c 9.81 2 12.5 68.1

(0) d 2 5 19.62 m/s For the next interval (from t 5 2 to 4 s), the computation is repeated, with the result

y 5 19.62 1 c9.81 2 12.5 68.1

(19.62)d2 5 32.04 m/s The calculation is continued in a similar fashion to obtain additional values:

t, s v, m/s

0 0.00 2 19.62 4 32.04 6 39.90 8 44.87 10 48.02 12 50.01 ` 53.44

The results are plotted in Fig. 1.5 along with the exact solution. It can be seen that the numerical method captures the essential features of the exact solution. However, be- cause we have employed straight-line segments to approximate a continuously curving function, there is some discrepancy between the two results. One way to minimize such discrepancies is to use a smaller step size. For example, applying Eq. (1.12) at l-s intervals results in a smaller error, as the straight-line segments track closer to the true solution. Using hand calculations, the effort associated with using smaller and smaller step sizes would make such numerical solutions impractical. However, with the aid of the computer, large numbers of calculations can be performed easily. Thus, you can accurately model the velocity of the falling parachutist without having to solve the differential equation exactly.

As in the previous example, a computational price must be paid for a more accurate numerical result. Each halving of the step size to attain more accuracy leads to a doubling

cha9792x_ch01_011-026.indd Page 17 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 17 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

of the number of computations. Thus, we see that there is a trade-off between accuracy and computational effort. Such trade-offs fi gure prominently in numerical methods and constitute an important theme of this book. Consequently, we have devoted the Epilogue of Part One to an introduction to more of these trade-offs.

1.2 CONSERVATION LAWS AND ENGINEERING

Aside from Newton’s second law, there are other major organizing principles in engineering. Among the most important of these are the conservation laws. Although they form the basis for a variety of complicated and powerful mathematical models, the great conserva- tion laws of science and engineering are conceptually easy to understand. They all boil down to

Change 5 increases 2 decreases (1.13)

This is precisely the format that we employed when using Newton’s law to develop a force balance for the falling parachutist [Eq. (1.8)]. Although simple, Eq. (1.13) embodies one of the most fundamental ways in which conservation laws are used in engineering—that is, to predict changes with respect to time. We give Eq. (1.13) the special name time-variable (or transient) computation. Aside from predicting changes, another way in which conservation laws are applied is for cases where change is nonexistent. If change is zero, Eq. (1.13) becomes

Change 5 0 5 increases 2 decreases

or

Increases 5 decreases (1.14)

0 0

20

40

4 8 12

t, s

v, m

/s

Terminal velocity

Exact, analytical solution

Approximate, numerical solution

FIGURE 1.5 Comparison of the numerical and analytical solutions for the falling parachutist problem.

cha9792x_ch01_011-026.indd Page 18 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 18 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

1.2 CONSERVATION LAWS AND ENGINEERING 19

Thus, if no change occurs, the increases and decreases must be in balance. This case, which is also given a special name—the steady-state computation—has many applica- tions in engineering. For example, for steady-state incompressible fl uid fl ow in pipes, the fl ow into a junction must be balanced by fl ow going out, as in

Flow in 5 fl ow out

For the junction in Fig. 1.6, the balance can be used to compute that the fl ow out of the fourth pipe must be 60. For the falling parachutist, steady-state conditions would correspond to the case where the net force was zero, or [Eq. (1.8) with dy�dt 5 0]

mg 5 cy (1.15)

Thus, at steady state, the downward and upward forces are in balance, and Eq. (1.15) can be solved for the terminal velocity

y 5 mg

c

Although Eqs. (1.13) and (1.14) might appear trivially simple, they embody the two fundamental ways that conservation laws are employed in engineering. As such, they will form an important part of our efforts in subsequent chapters to illustrate the connection between numerical methods and engineering. Our primary vehicles for making this con- nection are the engineering applications that appear at the end of each part of this book. Table 1.1 summarizes some of the simple engineering models and associated conserva- tion laws that will form the basis for many of these engineering applications. Most of the chemical engineering applications will focus on mass balances for reactors. The mass balance is derived from the conservation of mass. It specifi es that the change of mass of a chemical in the reactor depends on the amount of mass fl owing in minus the mass fl owing out. Both the civil and mechanical engineering applications will focus on models devel- oped from the conservation of momentum. For civil engineering, force balances are utilized to analyze structures such as the simple truss in Table 1.1. The same principles are employed for the mechanical engineering applications to analyze the transient up-and-down motion or vibrations of an automobile.

Pipe 2 Flow in = 80

Pipe 3 Flow out = 120

Pipe 4 Flow out = ?

Pipe 1 Flow in = 100

FIGURE 1.6 A fl ow balance for steady-state incompressible fl uid fl ow at the junction of pipes.

cha9792x_ch01_011-026.indd Page 19 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 19 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

TABLE 1.1 Devices and types of balances that are commonly used in the four major areas of engineering. For each case, the conservation law upon which the balance is based is specifi ed.

Structure

Civil engineering Conservation of momentum

Chemical engineering

Field Device Organizing Principle Mathematical Expression

Conservation of mass

Force balance:

Mechanical engineering Conservation of momentum

Machine Force balance:

Electrical engineering Conservation of charge Current balance:

Conservation of energy Voltage balance:

Mass balance: Reactors Input Output

Over a unit of time period �mass = inputs – outputs

At each node � horizontal forces (FH) = 0 � vertical forces (FV) = 0

For each node � current (i ) = 0

Around each loop � emf’s – � voltage drops for resistors = 0 � � – � iR = 0

– FV

+ FV

+ FH– FH

+ i2

– i3+ i1+

Circuit i1R1

i3R3

i2R2 �

Upward force

Downward force

x = 0

m = downward force – upward forced 2x

dt2

cha9792x_ch01_011-026.indd Page 20 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 20 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 21

TABLE 1.2 Some practical issues that will be explored in the engineering applications at the end of each part of this book.

1. Nonlinear versus linear. Much of classical engineering depends on linearization to permit analytical solutions. Although this is often appropriate, expanded insight can often be gained if nonlinear problems are examined.

2. Large versus small systems. Without a computer, it is often not feasible to examine systems with over three interacting components. With computers and numerical methods, more realistic multicomponent systems can be examined.

3. Nonideal versus ideal. Idealized laws abound in engineering. Often there are nonidealized alternatives that are more realistic but more computationally demanding. Approximate numerical approaches can facilitate the application of these nonideal relationships.

4. Sensitivity analysis. Because they are so involved, many manual calculations require a great deal of time and effort for successful implementation. This sometimes discourages the analyst from implementing the multiple computations that are necessary to examine how a system responds under different conditions. Such sensitivity analyses are facilitated when numerical methods allow the computer to assume the computational burden.

5. Design. It is often a straightforward proposition to determine the performance of a system as a function of its parameters. It is usually more diffi cult to solve the inverse problem—that is, determining the parameters when the required performance is specifi ed. Numerical methods and computers often permit this task to be implemented in an effi cient manner.

Finally, the electrical engineering applications employ both current and energy bal- ances to model electric circuits. The current balance, which results from the conservation of charge, is similar in spirit to the fl ow balance depicted in Fig. 1.6. Just as fl ow must balance at the junction of pipes, electric current must balance at the junction of electric wires. The energy balance specifi es that the changes of voltage around any loop of the circuit must add up to zero. The engineering applications are designed to illustrate how numerical methods are actually employed in the engineering problem-solving process. As such, they will permit us to explore practical issues (Table 1.2) that arise in real-world applications. Making these connections between mathematical techniques such as nu- merical methods and engineering practice is a critical step in tapping their true potential. Careful examination of the engineering applications will help you to take this step.

PROBLEMS

1.1 Use calculus to solve Eq. (1.9) for the case where the initial velocity, y(0) is nonzero. 1.2 Repeat Example 1.2. Compute the velocity to t 5 8 s, with a step size of (a) 1 and (b) 0.5 s. Can you make any statement regard- ing the errors of the calculation based on the results? 1.3 Rather than the linear relationship of Eq. (1.7), you might choose to model the upward force on the parachutist as a second- order relationship,

FU 5 2c¿y2

where c9 5 a bulk second-order drag coeffi cient (kg/m). (a) Using calculus, obtain the closed-form solution for the case

where the jumper is initially at rest (y 5 0 at t 5 0).

(b) Repeat the numerical calculation in Example 1.2 with the same initial condition and parameter values, but with second-order drag. Use a value of 0.22 kg/m for c9.

1.4 For the free-falling parachutist with linear drag, assume a fi rst jumper is 70 kg and has a drag coeffi cient of 12 kg/s. If a second jumper has a drag coeffi cient of 15 kg/s and a mass of 80 kg, how long will it take him to reach the same velocity the fi rst jumper reached in 9 s? 1.5 Compute the velocity of a free-falling parachutist using Euler’s method for the case where m 5 80 kg and c 5 10 kg/s. Perform the calculation from t 5 0 to 20 s with a step size of 1 s. Use an initial condition that the parachutist has an upward velocity of 20 m/s at t 5 0. At t 5 10 s, assume that the chute is instantaneously deployed so that the drag coeffi cient jumps to 60 kg/s.

cha9792x_ch01_011-026.indd Page 21 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 21 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

1.6 The following information is available for a bank account:

Date Deposits Withdrawals Interest Balance

5/1 1522.33 220.13 327.26

6/1 216.80 378.51

7/1 450.35 106.80

8/1 127.31 350.61

9/1

Note that the money earns interest which is computed as

Interest 5 i Bi

where i 5 the interest rate expressed as a fraction per month, and Bi the initial balance at the beginning of the month. (a) Use the conservation of cash to compute the balance on 6/1,

7/1, 8/1, and 9/1 if the interest rate is 1% per month (i 5 0.01/month). Show each step in the computation.

(b) Write a differential equation for the cash balance in the form

dB

dt 5 f (D(t), W(t), i)

where t 5 time (months), D(t) 5 deposits as a function of time ($/month), W(t) 5 withdrawals as a function of time ($/month). For this case, assume that interest is compounded continu- ously; that is, interest 5 iB.

(c) Use Euler’s method with a time step of 0.5 month to simulate the balance. Assume that the deposits and withdrawals are ap- plied uniformly over the month.

(d) Develop a plot of balance versus time for (a) and (c). 1.7 The amount of a uniformly distributed radioactive contaminant contained in a closed reactor is measured by its concentration c (becquerel/liter or Bq/L). The contaminant decreases at a decay rate proportional to its concentration—that is,

decay rate 5 2kc

where k is a constant with units of day21. Therefore, according to Eq. (1.13), a mass balance for the reactor can be written as

dc

dt 5 2kc

achange in mass

b 5 adecrease by decay

b (a) Use Euler’s method to solve this equation from t 5 0 to 1 d

with k 5 0.175d21. Employ a step size of Dt 5 0.1. The con- centration at t 5 0 is 100 Bq/L.

(b) Plot the solution on a semilog graph (i.e., ln c versus t) and determine the slope. Interpret your results.

1.8 A group of 35 students attend a class in a room that measures 11 m by 8 m by 3 m. Each student takes up about 0.075 m3 and gives out about 80 W of heat (1 W 5 1 J/s). Calculate the air tem- perature rise during the fi rst 20 minutes of the class if the room is completely sealed and insulated. Assume the heat capacity, Cy, for air is 0.718 kJ/(kg K). Assume air is an ideal gas at 208C and 101.325 kPa. Note that the heat absorbed by the air Q is related to the mass of the air m, the heat capacity, and the change in tempera- ture by the following relationship:

Q 5 m# T2

T1

CydT 5 mCy (T2 2 T1)

The mass of air can be obtained from the ideal gas law:

PV 5 m

MwT RT

where P is the gas pressure, V is the volume of the gas, Mwt is the molecular weight of the gas (for air, 28.97 kg/kmol), and R is the ideal gas constant [8.314 kPa m3/(kmol K)]. 1.9 A storage tank contains a liquid at depth y, where y 5 0 when the tank is half full. Liquid is withdrawn at a constant fl ow rate Q to meet demands. The contents are resupplied at a sinusoidal rate 3Q sin2(t).

y

0

FIGURE P1.9

Equation (1.13) can be written for this system as

d(Ay)

dt 5 3Q sin 2(t) 2 Q

achange in volume

b 5 (inflow)  2 (outflow) or, since the surface area A is constant

dy

dt 5 3

Q

A sin 2(t) 2

Q

A

cha9792x_ch01_011-026.indd Page 22 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 22 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 23

Use Euler’s method to solve for the depth y from t 5 0 to 10 d with a step size of 0.5 d. The parameter values are A 5 1250 m2 and Q 5 450 m3/d. Assume that the initial condition is y 5 0. 1.10 For the same storage tank described in Prob. 1.9, suppose that the outfl ow is not constant but rather depends on the depth. For this case, the differential equation for depth can be written as

dy

dt 5 3

Q

A sin 2(t) 2

a(1 1 y)1.5

A

Use Euler’s method to solve for the depth y from t 5 0 to 10 d with a step size of 0.5 d. The parameter values are A 5 1250 m2, Q 5 450 m3/d, and a 5 150. Assume that the initial condition is y 5 0. 1.11 Apply the conservation of volume (see Prob. 1.9) to simulate the level of liquid in a conical storage tank (Fig. P1.11). The liquid fl ows in at a sinusoidal rate of Qin 5 3 sin

2(t) and fl ows out accord- ing to

Qout 5 3(y 2 yout) 1.5 y . yout

Qout 5 0 y # yout

where fl ow has units of m3/d and y 5 the elevation of the water sur- face above the bottom of the tank (m). Use Euler’s method to solve for the depth y from t 5 0 to 10 d with a step size of 0.5 d. The pa- rameter values are rtop 5 2.5 m, ytop 5 4 m, and yout 5 1 m. Assume that the level is initially below the outlet pipe with y(0) 5 0.8 m.

ytop

y

yout

0

Qin

Qouts

1

rtop

FIGURE P1.11

1.12 In our example of the free-falling parachutist, we assumed that the acceleration due to gravity was a constant value. Although this is a decent approximation when we are examining falling objects near the surface of the earth, the gravitational force decreases as we move above sea level. A more general representation based on Newton’s inverse square law of gravitational attraction can be written as

g(x) 5 g(0) R2

(R 1 x)2

where g(x) 5 gravitational acceleration at altitude x (in m) mea- sured upward from the earth’s surface (m/s2), g(0) 5 gravitational acceleration at the earth’s surface (> 9.81 m/s2), and R 5 the earth’s radius (> 6.37 3 106 m). (a) In a fashion similar to the derivation of Eq. (1.9) use a force

balance to derive a differential equation for velocity as a func- tion of time that utilizes this more complete representation of gravitation. However, for this derivation, assume that upward velocity is positive.

(b) For the case where drag is negligible, use the chain rule to ex- press the differential equation as a function of altitude rather than time. Recall that the chain rule is

dy

dt 5

dy

dx dx

dt

(c) Use calculus to obtain the closed form solution where y 5 y0 at x 5 0.

(d) Use Euler’s method to obtain a numerical solution from x 5 0 to 100,000 m using a step of 10,000 m where the initial velocity is 1500 m/s upward. Compare your result with the analytical solution.

1.13 Suppose that a spherical droplet of liquid evaporates at a rate that is proportional to its surface area.

dV

dt 5 2kA

where V 5 volume (mm3), t 5 time (min), k 5 the evaporation rate (mm/min), and A 5 surface area (mm2). Use Euler’s method to compute the volume of the droplet from t 5 0 to 10 min using a step size of 0.25 min. Assume that k 5 0.08 mm/min and that the droplet initially has a radius of 2.5 mm. Assess the validity of your results by determining the radius of your fi nal computed volume and veri- fying that it is consistent with the evaporation rate. 1.14 Newton’s law of cooling says that the temperature of a body changes at a rate proportional to the difference between its temperature and that of the surrounding medium (the ambient temperature),

dT

dt 5 2k(T 2 Ta)

where T 5 the temperature of the body (8C), t 5 time (min), k 5 the proportionality constant (per minute), and Ta 5 the ambi- ent temperature (8C). Suppose that a cup of coffee originally has a temperature of 708C. Use Euler’s method to compute the temperature from t 5 0 to 10 min using a step size of 2 min if Ta 5 208C and k 5 0.019/min. 1.15 As depicted in Fig. P1.15, an RLC circuit consists of three elements: a resistor (R), and inductor (L) and a capacitor (C). The fl ow of current across each element induces a voltage drop.

cha9792x_ch01_011-026.indd Page 23 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 23 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

Q1

Q10 Q9 Q8

Q3 Q5

Q7Q6Q4Q2

FIGURE P1.17

1.18 The velocity is equal to the rate of change of distance x (m),

dx

dt 5 y(t) (P1.18)

(a) Substitute Eq. (1.10) and develop an analytical solution for distance as a function of time. Assume that x(0) 5 0.

(b) Use Euler’s method to numerically integrate Eqs. (P1.18) and (1.9) in order to determine both the velocity and distance fallen as a function of time for the fi rst 10 s of free-fall using the same parameters as in Example 1.2.

(c) Develop a plot of your numerical results together with the ana- lytical solution.

1.19 You are working as a crime-scene investigator and must pre- dict the temperature of a homicide victim over a 5-hr period. You know that the room where the victim was found was at 108C when the body was discovered. (a) Use Newton’s law of cooling (Prob. 1.14) and Euler’s method

to compute the victim’s body temperature for the 5-hr period using values of k 5 0.12/hr and Dt 5 0.5 hr. Assume that the victim’s body temperature at the time of death was 378C, and that the room temperature was at a constant value of 108C over the 5-hr period.

(b) Further investigation reveals that the room temperature had actually dropped linearly from 20 to 108C over the 5-hr period. Repeat the same calculation as in (a) but incorporate this new information.

(c) Compare the results from (a) and (b) by plotting them on the same graph.

1.20 Suppose that a parachutist with linear drag (m 5 70 kg, c 5 12.5 kg/s) jumps from an airplane fl ying at an altitude of a kilo- meter with a horizontal velocity of 180 m/s relative to the ground. (a) Write a system of four differential equations for x, y, yx 5 dx/dt

and yy 5 dy/dt.

Kirchhoff’s second voltage law states that the algebraic sum of these voltage drops around a closed circuit is zero,

iR 1 L di

dt 1

q

C 5 0

where i 5 current, R 5 resistance, L 5 inductance, t 5 time, q 5 charge, and C 5 capacitance. In addition, the current is related to charge as in

dq

dt 5 i

(a) If the initial values are i(0) 5 0 and q(0) 5 1 C, use Euler’s method to solve this pair of differential equations from t 5 0 to 0.1 s using a step size of Dt 5 0.01 s. Employ the following parameters for your calculation: R 5 200 V, L 5 5 H, and C 5 10–4 F.

(b) Develop a plot of i and q versus t.

q ciR

Resistor Inductor Capacitor

i

di dt

L

FIGURE P1.15

1.16 Cancer cells grow exponentially with a doubling time of 20 h when they have an unlimited nutrient supply. However, as the cells start to form a solid spherical tumor without a blood supply, growth at the center of the tumor becomes limited, and eventually cells start to die. (a) Exponential growth of cell number N can be expressed as

shown, where m is the growth rate of the cells. For cancer cells, fi nd the value of m.

dN

dt 5 mN

(b) Write an equation that will describe the rate of change of tumor volume during exponential growth given that the diameter of an individual cell is 20 microns.

(c) After a particular type of tumor exceeds 500 microns in diam- eter, the cells at the center of the tumor die (but continue to take up space in the tumor). Determine how long it will take for the tumor to exceed this critical size.

1.17 A fl uid is pumped into the network shown in Fig. P1.17. If Q2 5 0.6, Q3 5 0.4, Q7 5 0.2, and Q8 5 0.3 m

3/s, determine the other fl ows.

cha9792x_ch01_011-026.indd Page 24 09/10/13 7:29 PM F-468 cha9792x_ch01_011-026.indd Page 24 09/10/13 7:29 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 25

(b) At steady-state, use this equation to solve for the particle’s terminal velocity.

(c) Employ the result of (b) to compute the particle’s terminal velocity in m/s for a spherical silt particle settling in water: d 5 10 mm, r 5 1 g/cm3, rs 5 2.65 g/cm

3, and m 5 0.014 g/(cm?s). (d) Check whether fl ow is laminar. (e) Use Euler’s method to compute the velocity from t 5 0 to 2215 s

with Dt 5 2218 s given the parameters given previously along with the initial condition: y (0) 5 0.

FG

FD FB

d

FIGURE P1.22

1.23 As described in Prob. 1.22, in addition to the downward force of gravity (weight) and drag, an object falling through a fl uid is also subject to a buoyancy force that is proportional to the displaced volume. For example, for a sphere with diameter d (m), the sphere’s volume is V 5 pd3/6 and its projected area is A 5 pd2/4. The buoy- ancy force can then be computed as Fb 5 –rVg. We neglected buoyancy in our derivation of Eq. (1.9) because it is relatively small for an object like a parachutist moving through air. However, for a more dense fl uid like water, it becomes more prominent. (a) Derive a differential equation in the same fashion as Eq. (1.9),

but include the buoyancy force and represent the drag force as described in Prob. 1.21.

(b) Rewrite the differential equation from (a) for the special case of a sphere.

(c) Use the equation developed in (b) to compute the terminal velocity (i.e., for the steady-state case). Use the following parameter values for a sphere falling through water: sphere diameter 5 1 cm, sphere density 5 2700 kg/m3, water density 5 1000 kg/m3, and Cd 5 0.47.

(d) Use Euler’s method with a step size of Dt 5 0.03125 s to nu- merically solve for the velocity from t 5 0 to 0.25 s with an initial velocity of zero.

1.24 As depicted in Fig. P1.24, the downward defl ection y (m) of a cantilever beam with a uniform load w (kg/m) can be computed as

y 5 w

24EI (x4 2 4Lx3 1 6L2x2)

where x 5 distance (m), E 5 the modulus of elasticity 5 2 3 1011 Pa, I 5 moment of inertia 5 3.25 3 10–4 m4, w 5 10,000 N/m, and

(b) If the initial horizontal position is defi ned as x 5 0, use Euler’s methods with Dt 5 1 s to compute the jumper’s position over the fi rst 10 s.

(c) Develop plots of y versus t and y versus x. Use the plot to graphically estimate when and where the jumper would hit the ground if the chute failed to open.

1.21 As noted in Prob. 1.3, drag is more accurately represented as depending on the square of velocity. A more fundamental represen- tation of the drag force, which assumes turbulent conditions (i.e., a high Reynolds number), can be formulated as

Fd 5 2 1

2 rACdyZyZ

where Fd 5 the drag force (N), r 5 fl uid density (kg/m 3), A 5 the fron-

tal area of the object on a plane perpendicular to the direction of motion (m2), y 5 velocity (m/s), and Cd 5 a dimensionless drag coeffi cient. (a) Write the pair of differential equations for velocity and position

(see Prob. 1.18) to describe the vertical motion of a sphere with di- ameter d (m) and a density of rs (kg/km

3). The differential equation for velocity should be written as a function of the sphere’s diameter.

(b) Use Euler’s method with a step size of Dt 5 2 s to compute the posi- tion and velocity of a sphere over the fi rst 14 s. Employ the follow- ing parameters in your calculation: d 5 120 cm, r 5 1.3 kg/m3, rs 5 2700 kg/m

3, and Cd 5 0.47. Assume that the sphere has the initial conditions: x(0) 5 100 m and y(0) 5 –40 m/s.

(c) Develop a plot of your results (i.e., y and y versus t) and use it to graphically estimate when the sphere would hit the ground.

(d) Compute the value for the bulk second-order drag coeffi cient cd9 (kg/m). Note that, as described in Prob. 1.3, the bulk second- order drag coeffi cient is the term in the fi nal differential equa- tion for velocity that multiplies the term y ZyZ.

1.22 As depicted in Fig. P1.22, a spherical particle settling through a quiescent fl uid is subject to three forces: the downward force of gravity (FG), and the upward forces of buoyancy (FB) and drag (FD). Both the gravity and buoyancy forces can be computed with Newton’s second law with the latter equal to the weight of the displaced fl uid. For lami- nar fl ow, the drag force can be computed with Stokes’s law,

FD 5 3pmdy

where m 5 the dynamic viscosity of the fl uid (N s/m2), d 5 the particle diameter (m), and y 5 the particle’s settling velocity (m/s). Note that the mass of the particle can be expressed as the product of the particle’s volume and density rs (kg/m

3) and the mass of the dis- placed fl uid can be computed as the product of the particle’s volume and the fl uid’s density r (kg/m3). The volume of a sphere is pd3/6. In addition, laminar fl ow corresponds to the case where the dimension- less Reynolds number, Re, is less than 1, where Re 5 rdy/m. (a) Use a force balance for the particle to develop the differential

equation for dy/dt as a function of d, r, rs, and m.

cha9792x_ch01_011-026.indd Page 25 09/12/13 9:08 AM F-468 cha9792x_ch01_011-026.indd Page 25 09/12/13 9:08 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26 MATHEMATICAL MODELING AND ENGINEERING PROBLEM SOLVING

1.26 Beyond fl uids, Archimedes’ principle has proven useful in geology when applied to solids on the earth’s crust. Figure P1.26 depicts one such case where a lighter conical granite mountain “fl oats on” a denser basalt layer at the earth’s surface. Note that the part of the cone below the surface is formally referred to as a frus- tum. Develop a steady-state force balance for this case in terms of the following parameters: basalt’s density (rb), granite’s density (rg), the cone’s bottom radius (r), and the height above (h1) and below (h2) the earth’s surface.

H

Basalt

Granite

h1

h2

r1

r2

FIGURE P1.26

L 5 length 5 4 m. This equation can be differentiated to yield the slope of the downward defl ection as a function of x:

dy

dx 5

w

24EI (4x3 2 12Lx2 1 12L2x)

If y 5 0 at x 5 0, use this equation with Euler’s method (Dx 5 0.125 m) to compute the defl ection from x 5 0 to L. Develop a plot of your results along with the analytical solution computed with the fi rst equation.

y

w

x = 0 x = L

0

FIGURE P1.24 A cantilever beam.

1.25 Use Archimedes’ principle to develop a steady-state force bal- ance for a spherical ball of ice fl oating in seawater (Fig. P1.25). The force balance should be expressed as a third-order polynomial (cubic) in terms of height of the cap above the water line (h), the seawater’s density (rf), the ball’s density (rs), and the ball’s radius (r).

h

r

FIGURE P1.25

cha9792x_ch01_011-026.indd Page 26 09/12/13 9:08 AM F-468 cha9792x_ch01_011-026.indd Page 26 09/12/13 9:08 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2

27

Programming and Software

In Chap. 1, we used a net force to develop a mathematical model to predict the fall velocity of a parachutist. This model took the form of a differential equation,

dy

dt 5 g 2

c m

y

We also learned that a solution to this equation could be obtained by a simple numerical approach called Euler’s method,

yi11 5 yi 1 dyi dt ¢t

Given an initial condition, this equation can be implemented repeatedly to compute the velocity as a function of time. However, to obtain good accuracy, many small steps must be taken. This would be extremely laborious and time-consuming to implement by hand. However, with the aid of the computer, such calculations can be performed easily. So our next task is to fi gure out how to do this. The present chapter will introduce you to how the computer is used as a tool to obtain such solutions.

2.1 PACKAGES AND PROGRAMMING

Today, there are two types of software users. On one hand, there are those who take what they are given. That is, they limit themselves to the capabilities found in the software’s standard mode of operation. For example, it is a straightforward proposition to solve a system of linear equations or to generate a plot of x-y values with either Excel or MATLAB software. Because this usually involves a minimum of effort, most users tend to adopt this “vanilla” mode of operation. In addition, since the designers of these packages anticipate most typical user needs, many meaningful problems can be solved in this way. But what happens when problems arise that are beyond the standard capability of the tool? Unfortunately, throwing up your hands and saying, “Sorry boss, no can do!” is not acceptable in most engineering circles. In such cases, you have two alternatives. First, you can look for a different package and see if it is capable of solving the problem. That is one of the reasons we have chosen to cover both Excel and MATLAB in this book. As you will see, neither one is all encompassing and each has different

C H A P T E R 2

cha9792x_ch02_027-054.indd Page 27 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 27 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28 PROGRAMMING AND SOFTWARE

strengths. By being conversant with both, you will greatly increase the range of problems you can address. Second, you can grow and become a “power user” by learning to write Excel VBA1 macros or MATLAB M-fi les. And what are these? They are nothing more than computer programs that allow you to extend the capabilities of these tools. Because engineers should never be content to be tool limited, they will do whatever is necessary to solve their prob- lems. A powerful way to do this is to learn to write programs in the Excel and MATLAB environments. Furthermore, the programming skills required for macros and M-fi les are the same as those needed to effectively develop programs in languages like Fortran 90 or C. The major goal of the present chapter is to show you how this can be done. However, we do assume that you have been exposed to the rudiments of computer programming. Therefore, our emphasis here is on facets of programming that directly affect its use in engineering problem solving.

2.1.1 Computer Programs

Computer programs are merely a set of instructions that direct the computer to perform a certain task. Since many individuals write programs for a broad range of applications, most high-level computer languages, like Fortran 90 and C, have rich capabilities. Although some engineers might need to tap the full range of these capabilities, most merely require the ability to perform engineering-oriented numerical calculations. Looked at from this perspective, we can narrow down the complexity to a few programming topics. These are:

• Simple information representation (constants, variables, and type declarations). • Advanced information representation (data structure, arrays, and records). • Mathematical formulas (assignment, priority rules, and intrinsic functions). • Input/output. • Logical representation (sequence, selection, and repetition). • Modular programming (functions and subroutines).

Because we assume that you have had some prior exposure to programming, we will not spend time on the fi rst four of these areas. At best, we offer them as a checklist that covers what you will need to know to implement the programs that follow. However, we will devote some time to the last two topics. We emphasize logical representation because it is the single area that most infl uences an algorithm’s coherence and understandability. We include modular programming because it also contributes greatly to a program’s organization. In addition, modules provide a means to archive useful algorithms in a convenient format for subsequent applications.

2.2 STRUCTURED PROGRAMMING

In the early days of computers, programmers usually did not pay much attention to whether their programs were clear and easy to understand. Today, it is recognized that there are many benefi ts to writing organized, well-structured code. Aside from the obvious benefi t of making software much easier to share, it also helps generate much more effi cient

1VBA is the acronym for Visual Basic for Applications.

cha9792x_ch02_027-054.indd Page 28 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 28 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.2 STRUCTURED PROGRAMMING 29

program development. That is, well-structured algorithms are invariably easier to debug and test, resulting in programs that take a shorter time to develop, test, and update. Computer scientists have systematically studied the factors and procedures needed to develop high-quality software of this kind. In essence, structured programming is a set of rules that prescribe good style habits for the programmer. Although structured programming is fl exible enough to allow considerable creativity and personal expression, its rules impose enough constraints to render the resulting codes far superior to unstruc- tured versions. In particular, the fi nished product is more elegant and easier to understand. A key idea behind structured programming is that any numerical algorithm can be composed using the three fundamental control structures: sequence, selection, and rep- etition. By limiting ourselves to these structures, the resulting computer code will be clearer and easier to follow. In the following paragraphs, we will describe each of these structures. To keep this description generic, we will employ fl owcharts and pseudocode. A fl owchart is a visual or graphical representation of an algorithm. The fl owchart employs a series of blocks and arrows, each of which represents a particular operation or step in the algorithm (Fig. 2.1). The arrows represent the sequence in which the operations are implemented. Not everyone involved with computer programming agrees that fl owcharting is a productive endeavor. In fact, some experienced programmers do not advocate fl ow- charts. However, we feel that there are three good reasons for studying them. First, they are still used for expressing and communicating algorithms. Second, even if they are not employed routinely, there will be times when they will prove useful in planning, unraveling, or communicating the logic of your own or someone else’s program. Finally, and most important for our purposes, they are excellent pedagogical tools. From a

FIGURE 2.1 Symbols used in fl owcharts.

SYMBOL NAME

Terminal

Flowlines

Process

Input/output

Decision

Junction

Off-page connector

Count-controlled loop

FUNCTION

Represents the beginning or end of a program.

Represents the flow of logic. The humps on the horizontal arrow indicate that it passes over and does not connect with the vertical flowlines.

Represents calculations or data manipulations.

Represents inputs or outputs of data and information.

Represents a comparison, question, or decision that determines alternative paths to be followed.

Represents the confluence of flowlines.

Represents a break that is continued on another page.

Used for loops which repeat a prespecified number of iterations.

cha9792x_ch02_027-054.indd Page 29 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 29 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30 PROGRAMMING AND SOFTWARE

teaching perspective, they are ideal vehicles for visualizing some of the fundamental control structures employed in computer programming. An alternative approach to express an algorithm that bridges the gap between fl ow- charts and computer code is called pseudocode. This technique uses code-like statements in place of the graphical symbols of the fl owchart. We have adopted some style conventions for the pseudocode in this book. Keywords such as IF, DO, INPUT, etc., are capitalized, whereas the conditions, processing steps, and tasks are in lowercase. Additionally, the processing steps are indented. Thus the keywords form a “sandwich” around the steps to visually defi ne the extent of each control structure. One advantage of pseudocode is that it is easier to develop a program with it than with a fl owchart. The pseudocode is also easier to modify and share with others. However, because of their graphic form, fl owcharts sometimes are better suited for visualizing complex algorithms. In the present text, we will use fl owcharts for pedagogical purposes. Pseudocode will be our principal vehicle for communicating algorithms related to numerical methods.

2.2.1 Logical Representation

Sequence. The sequence structure expresses the trivial idea that unless you direct it otherwise, the computer code is to be implemented one instruction at a time. As in Fig. 2.2, the structure can be expressed generically as a fl owchart or as pseudocode.

Selection. In contrast to the step-by-step sequence structure, selection provides a means to split the program’s fl ow into branches based on the outcome of a logical condition. Figure 2.3 shows the two most fundamental ways for doing this. The single-alternative decision, or IF/THEN structure (Fig. 2.3a), allows for a detour in the program fl ow if a logical condition is true. If it is false, nothing happens and the program moves directly to the next statement following the ENDIF. The double-alternative decision, or IF/THEN/ELSE structure (Fig. 2.3b), behaves in the same manner for a true condition. However, if the condition is false, the program implements the code between the ELSE and the ENDIF.

FIGURE 2.2 (a) Flowchart and (b) pseudocode for the sequence structure.

Instruction1

Instruction2

Instruction3

Instruction4

Instruction1 Instruction2 Instruction3 Instruction4

(a) Flowchart (b) Pseudocode

cha9792x_ch02_027-054.indd Page 30 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 30 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.2 STRUCTURED PROGRAMMING 31

Although the IF/THEN and the IF/THEN/ELSE constructs are suffi cient to construct any numerical algorithm, two other variants are commonly used. Suppose that the ELSE clause of an IF/THEN/ELSE contains another IF/THEN. For such cases, the ELSE and the IF can be combined in the IF/THEN/ELSEIF structure shown in Fig. 2.4a. Notice how in Fig. 2.4a there is a chain or “cascade” of decisions. The fi rst one is the IF statement, and each successive decision is an ELSEIF statement. Going down the chain, the fi rst condition encountered that tests true will cause a branch to its correspond- ing code block followed by an exit of the structure. At the end of the chain of conditions, if all the conditions have tested false, an optional ELSE block can be included. The CASE structure is a variant on this type of decision making (Fig. 2.4b). Rather than testing individual conditions, the branching is based on the value of a single test expression. Depending on its value, different blocks of code will be implemented. In addition, an optional block can be implemented if the expression takes on none of the prescribed values (CASE ELSE).

Repetition. Repetition provides a means to implement instructions repeatedly. The resulting constructs, called loops, come in two “fl avors” distinguished by how they are terminated.

FIGURE 2.3 Flowchart and pseudocode for simple selection constructs. (a) Single-alternative selection (IF/THEN) and (b) double- alternative selection (IF/THEN/ELSE).

(a) Single-alternative structure (IF/THEN)

(b) Double-alternative structure (IF/THEN/ELSE)

Flowchart Pseudocode

IF condition THEN True block ENDIF

True Condition

?

True Block

IF condition THEN True block ELSE False block ENDIF

TrueFalse Condition

?

True BlockFalse Block

cha9792x_ch02_027-054.indd Page 31 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 31 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32 PROGRAMMING AND SOFTWARE

The fi rst and most fundamental type is called a decision loop because it terminates based on the result of a logical condition. Figure 2.5 shows the most generic type of decision loop, the DOEXIT construct, also called a break loop. This structure repeats until a logical condition is true. It is not necessary to have two blocks in this structure. If the fi rst block is not included, the structure is sometimes called a pretest loop because the logical test is performed before anything occurs. Alternatively, if the second block is omitted, it is

(a) Multialternative structure (IF/THEN/ELSEIF)

(b) CASE structure (SELECT or SWITCH)

Flowchart Pseudocode

SELECT CASE Test Expression CASE Value1 Block1 CASE Value2 Block2 CASE Value3 Block3 CASE ELSE Block4 END SELECT

Value1 Value2 Value3 Else

Test expression

Block1 Block2 Block3 Block4

IF condition1 THEN Block1 ELSEIF condition2 Block2 ELSEIF condition3 Block3 ELSE Block4 ENDIF

TrueFalse

True

True

Condition1 ?

False

Condition3 ?

False

Condition2 ?

Block1

Block2

Block3Block4

FIGURE 2.4 Flowchart and pseudocode for supplementary selection or branching constructs. (a) Multiple- alternative selection (IF/THEN/ELSEIF) and (b) CASE construct.

cha9792x_ch02_027-054.indd Page 32 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 32 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.2 STRUCTURED PROGRAMMING 33

called a posttest loop. Because both blocks are included, the general case in Fig. 2.5 is sometimes called a midtest loop. It should be noted that the DOEXIT loop was introduced in Fortran 90 in an effort to simplify decision loops. This control construct is a standard part of the Excel VBA macro language but is not standard in C or MATLAB, which use the so-called WHILE structure. Because we believe that the DOEXIT is superior, we have adopted it as our decision loop structure throughout this book. In order to ensure that our algorithms are directly implemented in both MATLAB and Excel, we will show how the break loop can be simulated with the WHILE structure later in this chapter (see Sec. 2.5). The break loop in Fig. 2.5 is called a logical loop because it terminates on a logical condition. In contrast, a count-controlled or DOFOR loop (Fig. 2.6) performs a specifi ed number of repetitions, or iterations. The count-controlled loop works as follows. The index (represented as i in Fig. 2.6) is a variable that is set at an initial value of start. The program then tests whether the

FIGURE 2.5 The DOEXIT or break loop.

False

True Condition

?

DO Block1 IF condition EXIT Block2 ENDDO

Flowchart Pseudocode

Block1

Block2

FIGURE 2.6 The count-controlled or DOFOR construct.

i = startTrue

False

i > finish ? i = i + step DOFOR i = start, finish, step

Block ENDDO

Flowchart Pseudocode

Block

cha9792x_ch02_027-054.indd Page 33 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 33 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

34 PROGRAMMING AND SOFTWARE

index is less than or equal to the fi nal value, fi nish. If so, it executes the body of the loop, and then cycles back to the DO statement. Every time the ENDDO statement is encountered, the index is automatically increased by the step. Thus the index acts as a counter. Then, when the index is greater than the fi nal value (fi nish), the computer auto- matically exits the loop and transfers control to the line following the ENDDO statement. Note that for nearly all computer languages, including those of Excel and MATLAB, if the step is omitted, the computer assumes it is equal to 1.2

The numerical algorithms outlined in the following pages will be developed exclu- sively from the structures outlined in Figs. 2.2 through 2.6. The following example illustrates the basic approach by developing an algorithm to determine the roots for the quadratic formula.

EXAMPLE 2.1 Algorithm for Roots of a Quadratic

Problem Statement. The roots of a quadratic equation

ax2 1 bx 1 c 5 0

can be determined with the quadratic formula,

x1 x2

5 2b 6 2 Zb2 2 4ac Z

2a (E2.1.1)

Develop an algorithm that does the following:

Step 1: Prompts the user for the coeffi cients, a, b, and c. Step 2: Implements the quadratic formula, guarding against all eventualities (for example,

avoiding division by zero and allowing for complex roots). Step 3: Displays the solution, that is, the values for x. Step 4: Allows the user the option to return to step 1 and repeat the process.

Solution. We will use a top-down approach to develop our algorithm. That is, we will successively refi ne the algorithm rather than trying to work out all the details the fi rst time around. To do this, let us assume for the present that the quadratic formula is foolproof regardless of the values of the coeffi cients (obviously not true, but good enough for now). A structured algorithm to implement the scheme is

DO INPUT a, b, c r1 5 (2b 1 SQRT(b2 2 4ac))y(2a) r2 5 (2b 2 SQRT(b2 2 4ac))y(2a) DISPLAY r1, r2 DISPLAY 'Try again? Answer yes or no' INPUT response IF response 5 'no' EXIT ENDDO

2A negative step can be used. In such cases, the loop terminates when the index is less than the fi nal value.

cha9792x_ch02_027-054.indd Page 34 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 34 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.2 STRUCTURED PROGRAMMING 35

A DOEXIT construct is used to implement the quadratic formula repeatedly as long as the condition is false. The condition depends on the value of the character variable response. If response is equal to ‘yes’ the calculation is implemented. If not, that is, response 5 ‘no’ the loop terminates. Thus, the user controls termination by inputting a value for response. Now although the above algorithm works for certain cases, it is not foolproof. Depend- ing on the values of the coeffi cients, the algorithm might not work. Here is what can happen:

• If a 5 0, an immediate problem arises because of division by zero. In fact, close inspection of Eq. (E2.1.1) indicates that two different cases can arise. That is,

If b fi 0, the equation reduces to a linear equation with one real root, 2cyb. If b 5 0, then no solution exists. That is, the problem is trivial.

• If a fi 0, two possible cases occur depending on the value of the discriminant, d 5 b2 2 4ac. That is,

If d $ 0, two real roots occur. If d , 0, two complex roots occur.

Notice how we have used indentation to highlight the decisional structure that underlies the mathematics. This structure then readily translates to a set of coupled IF/THEN/ELSE structures that can be inserted in place of the shaded statements in the previous code to give the fi nal algorithm:

DO INPUT a, b, c r1 5 0: r2 5 0: i1 5 0: i2 5 0 IF a 5 0 THEN IF b fi 0 THEN r1 5 2cyb ELSE DISPLAY "Trivial solution" ENDIF ELSE discr 5 b2 2 4 * a * c IF discr $ 0 THEN r1 5 (2b 1 Sqrt(discr))y(2 * a) r2 5 (2b 2 Sqrt(discr))y(2 * a) ELSE r1 5 2by(2 * a) r2 5 r1 i1 5 Sqrt(Abs(discr))y(2 * a) i2 5 2il ENDIF ENDIF DISPLAY r1, r2, i1, i2 DISPLAY 'Try again? Answer yes or no' INPUT response IF response 5 'no' EXIT ENDDO

cha9792x_ch02_027-054.indd Page 35 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 35 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

36 PROGRAMMING AND SOFTWARE

The approach in the foregoing example can be employed to develop an algorithm for the parachutist problem. Recall that, given an initial condition for time and velocity, the problem involved iteratively solving the formula

yi11 5 yi 1 dyi dt ¢t (2.1)

Now also remember that if we desired to attain good accuracy, we would need to employ small steps. Therefore, we would probably want to apply the formula repeatedly from the initial time to the fi nal time. Consequently, an algorithm to solve the problem would be based on a loop. For example, suppose that we started the computation at t 5 0 and wanted to predict the velocity at t 5 4 s using a time step of Dt 5 0.5 s. We would, therefore, need to apply Eq. (2.1) eight times, that is,

n 5 4

0.5 5 8

where n 5 the number of iterations of the loop. Because this result is exact, that is, the ratio is an integer, we can use a count-controlled loop as the basis for the algorithm. Here is an example of the pseudocode:

g 5 9.81 INPUT cd, m INPUT ti, vi, tf, dt t 5 ti v 5 vi n 5 (tf 2 ti) y dt DOFOR i 5 1 TO n dvdt 5 g 2 (cd y m) * v v 5 v 1 dvdt * dt t 5 t 1 dt ENDDO DISPLAY v

Although this scheme is simple to program, it is not foolproof. In particular, it will work only if the computation interval is evenly divisible by the time step.3 In order to cover such cases, a decision loop can be substituted in place of the shaded area in the previous pseudocode. The fi nal result is

g 5 9.81 INPUT cd, m INPUT ti, vi, tf, dt t 5 ti v 5 vi

3This problem is compounded by the fact that computers use base-2 number representation for their internal math. Consequently, some apparently evenly divisible numbers do not yield integers when the division is implemented on a computer. We will cover this in Chap. 3.

cha9792x_ch02_027-054.indd Page 36 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 36 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.3 MODULAR PROGRAMMING 37

h 5 dt DO IF t 1 dt . tf THEN h 5 tf 2 t ENDIF dvdt 5 g 2 (cd y m) * v v 5 v 1 dvdt * h t 5 t 1 h IF t $ tf EXIT ENDDO DISPLAY v

As soon as we enter the loop, we use an IF/THEN structure to test whether adding t 1 dt will take us beyond the end of the interval. If it does not, which would usually be the case at fi rst, we do nothing. If it does, we would need to shorten the interval by setting the variable step h to t f 2 t. By doing this, we guarantee that the next step falls exactly on t f. After we implement this fi nal step, the loop will terminate because the condition t $ t f will test true. Notice that before entering the loop, we assign the value of the time step, dt, to another variable, h. We create this dummy variable so that our routine does not change the given value of dt if and when we shorten the time step. We do this in anticipation that we might need to use the original value of dt somewhere else in the event that this code is integrated within a larger program. It should be noted that the algorithm is still not foolproof. For example, the user could have mistakenly entered a step size greater than the calculation interval, for example, t f 2 ti 5 5 and dt 5 20. Thus, you might want to include error traps in your code to catch such errors and to then allow the user to correct the mistake.

2.3 MODULAR PROGRAMMING

Imagine how diffi cult it would be to study a textbook that had no chapters, sections, or paragraphs. Breaking complicated tasks or subjects into more manageable parts is one way to make them easier to handle. In the same spirit, computer programs can be divided into small subprograms, or modules, that can be developed and tested separately. This approach is called modular programming. The most important attribute of modules is that they be as independent and self- contained as possible. In addition, they are typically designed to perform a specifi c, well-defi ned function and have one entry and one exit point. As such, they are usually short (generally 50 to 100 instructions in length) and highly focused. In standard high-level languages such as Fortran 90 or C, the primary programming element used to represent each module is the procedure. A procedure is a series of com- puter instructions that together perform a given task. Two types of procedures are com- monly employed: functions and subroutines. The former usually returns a single result, whereas the latter returns several. In addition, it should be mentioned that much of the programming related to software packages like Excel and MATLAB involves the development of subprograms. Hence,

cha9792x_ch02_027-054.indd Page 37 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 37 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

38 PROGRAMMING AND SOFTWARE

Excel macros and MATLAB functions are designed to receive some information, perform a calculation, and return results. Thus, modular thinking is also consistent with how programming is implemented in package environments. Modular programming has a number of advantages. The use of small, self-contained units makes the underlying logic easier to devise and to understand for both the developer and the user. Development is facilitated because each module can be perfected in isolation. In fact, for large projects, different programmers can work on individual parts. Modular design also increases the ease with which a program can be debugged and tested because errors can be more easily isolated. Finally, program maintenance and modifi cation are facilitated. This is primarily due to the fact that new modules can be developed to perform additional tasks and then easily incorporated into the already coherent and organized scheme. While all these attributes are reason enough to use modules, the most important reason related to numerical engineering problem solving is that they allow you to main- tain your own library of useful modules for later use in other programs. This will be the philosophy of this book: All the algorithms will be presented as modules. This approach is illustrated in Fig. 2.7, which shows a function developed to imple- ment Euler’s method. Notice that this function application and the previous versions differ in how they handle input/output. In the former versions, input and output directly come from (via INPUT statements) and to (via DISPLAY statements) the user. In the function, the inputs are passed into the FUNCTION via its argument list

Function Euler(dt, ti, tf, yi)

and the output is returned via the assignment statement

y 5 Euler(dt, ti, tf, yi)

In addition, recognize how generic the routine has become. There are no references to the specifi cs of the parachutist problem. For example, rather than calling the dependent

FUNCTION Euler(dt, ti, tf, yi) t 5 ti y 5 yi h 5 dt DO IF t 1 dt . tf THEN h 5 tf 2 t ENDIF dydt 5 dy(t, y) y 5 y 1 dydt * h t 5 t 1 h IF t $ tf EXIT ENDDO Euler 5 y END Euler

FIGURE 2.7 Pseudocode for a function that solves a differential equation using Euler’s method.

cha9792x_ch02_027-054.indd Page 38 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 38 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.4 EXCEL 39

variable y for velocity, the more generic label, y, is used within the function. Further, notice that the derivative is not computed within the function by an explicit equation. Rather, another function, dy, must be invoked to compute it. This acknowledges the fact that we might want to use this function for many different problems beyond solving for the parachutist’s velocity.

2.4 EXCEL

Excel is the spreadsheet produced by Microsoft, Inc. Spreadsheets are a special type of mathematical software that allow the user to enter and perform calculations on rows and columns of data. As such, they are a computerized version of a large accounting work- sheet on which large interconnected calculations can be implemented and displayed. Because the entire calculation is updated when any value on the sheet is changed, spread- sheets are ideal for “what if?” sorts of analysis. Excel has some built-in numerical capabilities including equation solving, curve fi tting, and optimization. It also includes VBA as a macro language that can be used to implement numerical calculations. Finally, it has several visualization tools, such as graphs and three-dimensional surface plots, that serve as valuable adjuncts for numerical analysis. In the present section, we will show how these capabilities can be used to solve the parachutist problem. To do this, let us fi rst set up a simple spreadsheet. As shown below, the fi rst step involves entering labels and numbers into the spreadsheet cells.

Before we write a macro program to calculate the numerical value, we can make our subsequent work easier by attaching names to the parameter values. To do this, select cells A3:B5 (the easiest way to do this is by moving the mouse to A3, holding down the left mouse button and dragging down to B5). Next, go to the Formulas tab and in the Defi ned Names group, click Create from Selection. This will open the Create Names from Selection dialog box, where the Left column box should be automatically selected. Then click OK to create the names. To verify that this has worked properly, select cell B3 and check that the label “m” appears in the name box (located on the left side of the sheet just below the menu bars).

cha9792x_ch02_027-054.indd Page 39 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 39 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

40 PROGRAMMING AND SOFTWARE

Move to cell C8 and enter the analytical solution (Eq. 1.9),

=9.81*m/cd*(1−exp(−cd/m*A8))

When this formula is entered, the value 0 should appear in cell C8. Then copy the for- mula down to cell C9 to give a value of 16.405 m/s. All the above is typical of the standard use of Excel. For example, at this point you could change parameter values and see how the analytical solution changes. Now, we will illustrate how VBA macros can be used to extend the standard capa- bilities. Figure 2.8 lists pseudocode alongside Excel VBA code for all the control struc- tures described in Sec. 2.2 (Figs. 2.3 through 2.6). Notice how, although the details differ, the structure of the pseudocode and the VBA code are identical. We can now use some of the constructs from Fig. 2.8 to write a macro function to numerically compute velocity. Open VBA by selecting4

Tools Macro Visual Basic Editor

Once inside the Visual Basic Editor (VBE), select

Insert Module

and a new code window will open up. The following VBA function can be developed directly from the pseudocode in Fig. 2.7. Type it into the code window.

Option Explicit

Function Euler(dt, ti, tf, yi, m, cd)

Dim h As Double, t As Double, y As Double, dydt As Double t = ti y = yi h = dt Do If t + dt > tf Then h = tf − t End If dydt = dy(t, y, m, cd) y = y + dydt * h t = t + h If t >= tf Then Exit Do Loop Euler = y End Function

Compare this macro with the pseudocode from Fig. 2.7 and recognize how similar they are. Also, see how we have expanded the function’s argument list to include the necessary parameters for the parachutist velocity model. The resulting velocity, y, is then passed back to the spreadsheet via the function name.

4The hot key combination Alt-F11 is even quicker!

cha9792x_ch02_027-054.indd Page 40 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 40 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

41

(a) Pseudocode

IF/THEN: IF condition THEN True block ENDIF

IF/THEN/ELSE: IF condition THEN True block ELSE False block ENDIF

IF/THEN/ELSEIF: IF condition1 THEN Block1 ELSEIF condition2 Block2 ELSEIF condition3 Block3 ELSE Block4 ENDIF

CASE: SELECT CASE Test Expression CASE Value1 Block1 CASE Value2 Block2 CASE Value3 Block3 CASE ELSE Block4 END SELECT

DOEXIT: DO Block1 IF condition EXIT Block2 ENDDO

COUNT-CONTROLLED LOOP: DOFOR i = start, finish, step Block ENDDO

(b) Excel VBA

If b <> 0 Then r1 = −c / b End If

If a < 0 Then b = Sqr(Abs(a)) Else b = Sqr(a) End If

If class = 1 Then x = x + 8 ElseIf class < 1 Then x = x − 8 ElseIf class < 10 Then x = x − 32 Else x = x − 64 End If

Select Case a + b Case Is < −50 x = −5 Case Is < 0 x = −5 − (a + b) / 10 Case Is < 50 x = (a + b) / 10 Case Else x = 5 End Select

Do i = i + 1 If i >= 10 Then Exit Do j = i*x Loop

For i = 1 To 10 Step 2 x = x + i Next i

FIGURE 2.8 The fundamental control structures in (a) pseudocode and (b) Excel VBA.

cha9792x_ch02_027-054.indd Page 41 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 41 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

42 PROGRAMMING AND SOFTWARE

Also notice how we have included another function to compute the derivative. This can be entered in the same module by typing it directly below the Euler function,

Function dy(t, v, m, cd) Const g As Double = 9.81 dy = g − (cd / m) * v End Function

The fi nal step is to return to the spreadsheet and invoke the function by entering the following formula in cell B9

=Euler(dt,A8,A9,B8,m,cd)

The result of the numerical integration, 16.531, will appear in cell B9. You should appreciate what has happened here. When you enter the function into the spreadsheet cell, the parameters are passed into the VBA program where the calcula- tion is performed and the result is then passed back and displayed in the cell. In effect, the VBA macro language allows you to use Excel as your input/output mechanism. All sorts of benefi ts arise from this fact. For example, now that you have set up the calculation, you can play with it. Suppose that the jumper was much heavier, say, m 5 100 kg (about 220 lb). Enter 100 into cell B3 and the spreadsheet will update immediately to show a value of 17.438 in cell B9. Change the mass back to 68.1 kg and the previous result, 16.531, automatically reappears in cell B9. Now let us take the process one step further by fi lling in some additional numbers for the time. Enter the numbers 4, 6, . . . 16 in cells A10 through A16. Then copy the formu- las from cells B9:C9 down to rows 10 through 16. Notice how the VBA program calculates the numerical result correctly for each new row. (To verify this, change dt to 2 and compare with the results previously computed by hand in Example 1.2.) An additional embellish- ment would be to develop an x-y plot of the results using the Excel Chart Wizard. The fi nal spreadsheet is shown below. We now have created a pretty nice problem- solving tool. You can perform sensitivity analyses by changing the values for each of

cha9792x_ch02_027-054.indd Page 42 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 42 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.5 MATLAB 43

the parameters. As each new value is entered, the computation and the graph would be automatically updated. It is this interactive nature that makes Excel so powerful. How- ever, recognize that the ability to solve this problem hinges on being able to write the macro with VBA. It is the combination of the Excel environment with the VBA programming language that truly opens up a world of possibilities for engineering problem solving. In the com- ing chapters, we will illustrate how this is accomplished.

2.5 MATLAB

MATLAB is the fl agship software product of The MathWorks, Inc., which was cofounded by the numerical analysts Cleve Moler and John N. Little. As the name implies, MATLAB was originally developed as a matrix laboratory. To this day, the major element of MAT- LAB is still the matrix. Mathematical manipulations of matrices are very conveniently implemented in an easy-to-use, interactive environment. To these matrix manipulations, MATLAB has added a variety of numerical functions, symbolic computations, and visu- alization tools. As a consequence, the present version represents a fairly comprehensive technical computing environment. MATLAB has a variety of functions and operators that allow convenient implemen- tation of many of the numerical methods developed in this book. These will be described in detail in the individual chapters that follow. In addition, programs can be written as so-called M-fi les that can be used to implement numerical calculations. Let us explore how this is done. First, you should recognize that normal MATLAB use is closely related to program- ming. For example, suppose that we wanted to determine the analytical solution to the parachutist problem. This could be done with the following series of MATLAB commands

>> g=9.81; >> m=68.1; >> cd=12.5; >> tf=2; >> v=g*m/cd*(1−exp(−cd/m*tf))

with the result being displayed as

v = 16.4217

Thus, the sequence of commands is just like the sequence of instructions in a typical programming language. Now what if you want to deviate from the sequential structure. Although there are some neat ways to inject some nonsequential capabilities in the standard command mode, the inclusion of decisions and loops is best done by creating a MATLAB document called an M-fi le. To do this, make the menu selection

File New Script

cha9792x_ch02_027-054.indd Page 43 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 43 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

44 PROGRAMMING AND SOFTWARE

and a new window will open with a heading “MATLAB Editor/Debugger.” In this window, you can type and edit MATLAB programs. Type the following code there:

g=9.81; m=68.1; cd=12.5; tf=2; v=g*m/cd*(1−exp(−cd/m*tf))

Notice how the commands are written in exactly the way as they would be written in the front end of MATLAB. Save the program with the name: analpara. MATLAB will automatically attach the extension .m to denote it as an M-fi le: analpara.m. To run the program, you must go back to the command mode. The most direct way to do this is to click on the “MATLAB Command Window” button on the task bar (which is usually at the bottom of the screen). The program can now be run by typing the name of the M-fi le, analpara, which should look like

>> analpara

If you have done everything correctly, MATLAB should respond with the correct answer:

v = 16.4217

Now one problem with the foregoing is that it is set up to compute one case only. You can make it more fl exible by having the user input some of the variables. For example, suppose that you wanted to assess the impact of mass on the velocity at 2 s. The M-fi le could be rewritten as the following to accomplish this

g=9.81; m=input('mass (kg): '); cd=12.5; tf=2; v=g*m/cd*(1−exp(−cd/m*tf))

Save this as analpara2.m. If you typed analpara2 while being in command mode, the prompt would show

mass (kg):

The user could then enter a value like 100, and the result will be displayed as

v = 17.3597

Now it should be pretty clear how we can program a numerical solution with an M-fi le. In order to do this, we must fi rst understand how MATLAB handles logical and looping structures. Figure 2.9 lists pseudocode alongside MATLAB code for all the

cha9792x_ch02_027-054.indd Page 44 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 44 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.5 MATLAB 45

(a) Pseudocode

IF/THEN: IF condition THEN True block ENDIF

IF/THEN/ELSE: IF condition THEN True block ELSE False block ENDIF

IF/THEN/ELSEIF: IF condition1 THEN Block1 ELSEIF condition2 Block2 ELSEIF condition3 Block3 ELSE Block4 ENDIF

CASE: SELECT CASE Test Expression CASE Value1 Block1 CASE Value2 Block2 CASE Value3 Block3 CASE ELSE Block4 END SELECT

DOEXIT: DO Block1 IF condition EXIT Block2 ENDDO

COUNT-CONTROLLED LOOP: DOFOR i = start, finish, step Block ENDDO

(b) MATLAB

if b ~= 0 r1 = −c / b; end

if a < 0 b = sqrt(abs(a)); else b 5 sqrt(a); end

if class == 1 x = x + 8; elseif class < 1 x = x − 8; elseif class < 10 x = x − 32; else x = x − 64; end

switch a + b case 1 x = −25; case 2 x = −5 − (a + b) / 10; case 3 x = (a + b) / 10; otherwise x = 5; end

while (1) i = i + 1; if i >= 10, break, end j = i*x; end

for i = 1:2:10 x = x + i; end

FIGURE 2.9 The fundamental control structures in (a) pseudocode and (b) the MATLAB program- ming language.

cha9792x_ch02_027-054.indd Page 45 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 45 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

46 PROGRAMMING AND SOFTWARE

control structures from Sec. 2.2. Although the structures of the pseudocode and the MATLAB code are very similar, there are some slight differences that should be noted. In particular, look at how we have represented the DOEXIT structure. In place of the DO, we use the statement WHILE(1). Because MATLAB interprets the number 1 as corresponding to “true,” this statement will repeat infi nitely in the same manner as the DO statement. The loop is terminated with a break command. This command transfers control to the statement following the end statement that terminates the loop. Also notice that the parameters of the count-controlled loop are ordered differently. For the pseudocode, the loop parameters are specifi ed as start, finish, step. For MAT- LAB, the parameters are ordered as start:step:finish. The following MATLAB M-fi le can now be developed directly from the pseudocode in Fig. 2.7. Type it into the MATLAB Editor/Debugger:

g=9.81; m=input('mass (kg): '); cd=12.5; ti=0; tf=2; vi=0; dt=0.1; t = ti; v = vi; h = dt; while (1) if t + dt > tf h = tf − t; end dvdt = g − (cd / m) * v; v = v + dvdt * h; t = t + h; if t >= tf, break, end end disp('velocity (m/s):') disp(v)

Save this fi le as numpara.m and return to the command mode and run it by entering: numpara. The following output should result:

mass (kg): 100

velocity (m/s): 17.4559

As a fi nal step in this development, let us take the above M-fi le and convert it into a proper function. This can be done in the following M-fi le based on the pseudocode from Fig. 2.7

function yy 5 euler(dt,ti,tf,yi,m,cd) t = ti; y = yi; h = dt;

cha9792x_ch02_027-054.indd Page 46 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 46 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

2.6 MATHCAD 47

while (1) if t + dt > tf h = tf − t; end dydt = dy(t, y, m, cd); y = y + dydt * h; t = t + h; if t >= tf, break, end end yy = y;

Save this fi le as euler.m and then create another M-fi le to compute the derivative,

function dydt = dy(t, v, m, cd) g = 9.81; dydt = g − (cd / m) * v;

Save this fi le as dy.m and return to the command mode. In order to invoke the function and see the result, you can type in the following commands

>> m=68.1; >> cd=12.5; >> ti=0; >> tf=2.; >> vi=0; >> dt=0.1; >> euler(dt,ti,tf,vi,m,cd)

When the last command is entered, the answer will be displayed as

ans = 16.5478

It is the combination of the MATLAB environment with the M-fi le programming language that truly opens up a world of possibilities for engineering problem solving. In the coming chapters we will illustrate how this is accomplished.

2.6 MATHCAD

Mathcad attempts to bridge the gap between spreadsheets like Excel and notepads. It was originally developed by Allen Razdow of MIT who cofounded Mathsoft, Inc., which published the fi rst commercial version in 1986. Today, Mathsoft is part of Parametric Technology Corporation (PTC) and Mathcad is in version 15. Mathcad is essentially an interactive notepad that allows engineers and scientists to perform a number of common mathematical, data-handling, and graphical tasks. Informa- tion and equations are input to a “whiteboard” design environment that is similar in spirit to a page of paper. Unlike a programming tool or spreadsheet, Mathcad’s interface accepts and displays natural mathematical notation using keystrokes or menu palette clicks—with no programming required. Because the worksheets contain live calculations, a single keystroke that changes an input or equation instantly returns an updated result.

cha9792x_ch02_027-054.indd Page 47 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 47 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

48 PROGRAMMING AND SOFTWARE

Mathcad can perform tasks in either numeric or symbolic mode. In numeric mode, Mathcad functions and operators give numerical responses, whereas in symbolic mode results are given as general expressions or equations. Maple V, a comprehensive symbolic math package, is the basis of the symbolic mode and was incorporated into Mathcad in 1993. Mathcad has a variety of functions and operators that allow convenient implementa- tion of many of the numerical methods developed in this book. These will be described in detail in succeeding chapters. In the event that you are unfamiliar with Mathcad, Appendix C also provides a primer on using this powerful software.

2.7 OTHER LANGUAGES AND LIBRARIES

In Secs. 2.4 and 2.5, we showed how Excel and MATLAB function procedures for Euler’s method could be developed from an algorithm expressed as pseudocode. You should recognize that similar functions can be written in high-level languages like Fortran 90 and C++. For example, a Fortran 90 function for Euler’s method is

Function Euler(dt, ti, tf, yi, m, cd)

REAL dt, ti, tf, yi, m, cd Real h, t, y, dydt

t = ti y = yi h = dt Do If (t + dt > tf) Then h = tf − t End If dydt = dy(t, y, m, cd) y = y + dydt * h t = t + h If (t >= tf) Exit End Do Euler = y End Function

For C, the result would look quite similar to the MATLAB function. The point is that once a well-structured algorithm is developed in pseudocode form, it can be readily implemented in a variety of programming environments. In this book, our approach will be to provide you with well-structured procedures written as pseudocode. This collection of algorithms then constitutes a numerical library that can be accessed to perform specifi c numerical tasks in a range of software tools and programming languages. Beyond your own programs, you should be aware that commercial programming libraries contain many useful numerical procedures. For example, the Numerical Recipe library includes a large range of algorithms written in Fortran and C.5 These procedures are described in both book (for example, Press et al. 2007) and electronic form.

5Numerical Recipe procedures are also available in book and electronic format for Pascal, MS BASIC, and MATLAB. Information on all the Numerical Recipe products can be found at http://www.nr.com/.

cha9792x_ch02_027-054.indd Page 48 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 48 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 49

2.4 The sine function can be evaluated by the following infi nite series:

sin x 5 x 2 x3

3! 1

x5

5! 2

x7

7! 1 p

Write an algorithm to implement this formula so that it computes and prints out the values of sin x as each term in the series is added. In other words, compute and print in sequence the values for

sin x 5 x

sin x 5 x 2 x3

3!

sin x 5 x 2 x3

3! 1

x5

5!

up to the order term n of your choosing. For each of the preceding, compute and display the percent relative error as

% error 5 true 2 series approximation

true 3 100%

Write the algorithm as (a) a structured fl owchart and (b) pseudocode. 2.5 Develop, debug, and document a program for Prob. 2.4 in either a high-level language or a macro language of your choice. Employ the library function for the sine in your computer to determine the true value. Have the program print out the series approximation and the error at each step. As a test case, employ the program to compute sin(1.5) for up to and including the term x15/15!. Interpret your results. 2.6 The following algorithm is designed to determine a grade for a course that consists of quizzes, homework, and a fi nal exam:

Step 1: Input course number and name. Step 2: Input weighting factors for quizzes (WQ), homework

(WH), and the fi nal exam (WF). Step 3: Input quiz grades and determine an average quiz grade (AQ). Step 4: Input homework grades and determine an average home-

work grade (AH). Step 5: If this course has a fi nal grade, continue to step 6. If not, go

to step 9. Step 6: Input fi nal exam grade (FE). Step 7: Determine average grade AG according to

AG 5 WQ 3 AQ 1 WH 3 AH 1 WF 3 FE

WQ 1 WH 1 WF 3 100%

Step 8: Go to step 10. Step 9: Determine average grade AG according to

AG 5 WQ 3 AQ 1 WH 3 AH

WQ 1 WH 3 100%

2.1 Write pseudocode to implement the fl owchart depicted in Fig. P2.1. Make sure that proper indentation is included to make the structure clear.

F

F

F

T

T

T

x = 75 x = 0

x = x – 50

x ≤ 500

x < 50

x < 100

FIGURE P2.1

2.2 Rewrite the following pseudocode using proper indentation

DO j 5 j 1 1 x 5 x 1 5 IF x . 5 THEN y 5 x ELSE y 5 0 ENDIF z 5 x 1 y IF z . 50 EXIT ENDDO

2.3 Develop, debug, and document a program to determine the roots of a quadratic equation, ax2 1 bx 1 c, in either a high-level language or a macro language of your choice. Use a subroutine procedure to compute the roots (either real or complex). Perform test runs for the cases (a) a 5 1, b 5 6, c 5 2; (b) a 5 0, b 5 24, c 5 1.6; (c) a 5 3, b 5 2.5, c 5 7.

PROBLEMS

cha9792x_ch02_027-054.indd Page 49 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 49 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

50 PROGRAMMING AND SOFTWARE

2.8 An amount of money P is invested in an account where interest is compounded at the end of the period. The future worth F yielded at an interest rate i after n periods may be determined from the following formula:

F 5 P(1 1 i)n

Write a program that will calculate the future worth of an investment for each year from 1 through n. The input to the function should include the initial investment P, the interest rate i (as a decimal), and the number of years n for which the future worth is to be calcu- lated. The output should consist of a table with headings and columns for n and F. Run the program for P 5 $100,000, i 5 0.04, and n 5 11 years. 2.9 Economic formulas are available to compute annual payments for loans. Suppose that you borrow an amount of money P and agree to repay it in n annual payments at an interest rate of i. The formula to compute the annual payment A is

A 5 P i(1 1 i)n

(1 1 i)n 2 1

Write a program to compute A. Test it with P 5 $55,000 and an interest rate of 6.6% (i 5 0.066). Compute results for n 5 1, 2, 3, 4, and 5 and display the results as a table with headings and columns for n and A. 2.10 The average daily temperature for an area can be approxi- mated by the following function,

T 5 Tmean 1 (Tpeak 2 Tmean) cos (v(t 2 tpeak) )

where Tmean 5 the average annual temperature, Tpeak 5 the peak temperature, v 5 the frequency of the annual variation (5 2p/365), and tpeak 5 day of the peak temperature (˘ 205 d). Develop a program that computes the average temperature between two days of the year for a particular city. Test it for (a) January–February (t 5 0 to 59) in Miami, Florida (Tmean 5 22.18C; Tpeak 5 28.38C), and (b) July–August (t 5 180 to 242) in Boston, Massachusetts (Tmean 5 10.78C; Tpeak 5 22.98C). 2.11 Develop, debug, and test a program in either a high-level language or a macro language of your choice to compute the velocity of the falling parachutist as outlined in Example 1.2. Design the program so that it allows the user to input values for the drag coeffi cient and mass. Test the program by duplicating the results from Example 1.2. Repeat the computation but em- ploy step sizes of 1  and 0.5 s. Compare your results with the analytical solution obtained previously in Example 1.1. Does a smaller step size make the results better or worse? Explain your results. 2.12 The bubble sort is an ineffi cient, but easy-to-program, sorting technique. The idea behind the sort is to move down through an array comparing adjacent pairs and swapping the

Step 10: Print out course number, name, and average grade. Step 11: Terminate computation. (a) Write well-structured pseudocode to implement this algorithm. (b) Write, debug, and document a structured computer program

based on this algorithm. Test it using the following data to calculate a grade without the fi nal exam and a grade with the fi nal exam: WQ 5 30; WH 5 40; WF 5 30; quizzes 5 98, 95, 90, 60, 99; homework 5 98, 95, 86, 100, 100, 77; and fi nal exam 5 91.

2.7 The “divide and average” method, an old-time method for approximating the square root of any positive number a can be formulated as

x 5 x 1 ayx

2

(a) Write well-structured pseudocode to implement this algorithm as depicted in Fig. P2.7. Use proper indentation so that the structure is clear.

(b) Develop, debug, and document a program to implement this equation in either a high-level language or a macro language of your choice. Structure your code according to Fig. P2.7.

F

F

T

T

SquareRoot = 0

SquareRoot = x

y = (x + a/x)/2 e = |(y – x)/y|

x = y

tol = 10�6

x = a/2

a > 0

e < tol

FIGURE P2.7

cha9792x_ch02_027-054.indd Page 50 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 50 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 51

decisional control structures (like If/Then, ElseIf, Else, End If). Design the function so that it returns the volume for all cases where the depth is less than 3R. Return an error message (“Overtop”) if you overtop the tank, that is, d . 3R. Test it with the following data:

R 1 1 1 1

d 0.5 1.2 3.0 3.1

2R

R

d

FIGURE P2.13

III

III IV

r

x

y

FIGURE P2.14

2.14 Two distances are required to specify the location of a point relative to an origin in two-dimensional space (Fig. P2.14):

• The horizontal and vertical distances (x, y) in Cartesian coordinates

• The radius and angle (r, u) in radial coordinates.

values if they are out of order. For this method to sort the array completely, it may need to pass through it many times. As the passes proceed for an ascending-order sort, the smaller elements in the array appear to rise toward the top like bubbles. Eventu- ally, there will be a pass through the array where no swaps are required. Then, the array is sorted. After the fi rst pass, the larg- est value in the array drops directly to the bottom. Consequently, the second pass only has to proceed to the second-to-last value, and so on. Develop a program to set up an array of 20 random numbers and sort them in ascending order with the bubble sort (Fig. P2.12).

TT

T

F

F

F

m = n – 1

switch = false

switch = truem = m – 1

i = 1

i = i + 1 i > m

swap ai ai+1

start

end

ai > ai+1 Not

switch

FIGURE P2.12

2.13 Figure P2.13 shows a cylindrical tank with a conical base. If the liquid level is quite low in the conical part, the volume is simply the conical volume of liquid. If the liquid level is mid- range in the cylindrical part, the total volume of liquid includes the fi lled conical part and the partially fi lled cylindrical part. Write a well-structured function procedure to compute the tank’s volume as a function of given values of R and d. Use

cha9792x_ch02_027-054.indd Page 51 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 51 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

52 PROGRAMMING AND SOFTWARE

Letter Criteria

A 90 # numeric grade # 100 B 80 # numeric grade , 90 C 70 # numeric grade , 80 D 60 # numeric grade , 70 F numeric grade , 60

2.16 Develop well-structured function procedures to determine (a) the factorial; (b) the minimum value in a vector; and (c) the average of the values in a vector. 2.17 Develop well-structured programs to (a) determine the square root of the sum of the squares of the elements of a two-dimensional array (i.e., a matrix) and (b) normalize a matrix by dividing each row by the maximum absolute value in the row so that the maxi- mum element in each row is 1. 2.18 Piecewise functions are sometimes useful when the relation- ship between a dependent and an independent variable cannot be adequately represented by a single equation. For example, the velocity of a rocket might be described by

y(t) 5 e 11t 2 2 5t 0 # t # 10

1100 2 5t 10 # t # 20

50t 1 2(t 2 20)2 20 # t # 30

1520e20.2(t230) t . 30

0 otherwise

Develop a well-structured function to compute v as a function of t. Then use this function to generate a table of v versus t for t 5 25 to 50 at increments of 0.5. 2.19 Develop a well-structured function to determine the elapsed days in a year. The function should be passed three values: mo 5 the month (1–12), da 5 the day (1–31) and leap 5 (0 for non–leap year and 1 for leap year). Test it for January 1, 1999; February 29, 2000; March 1, 2001; June 21, 2002; and December 31, 2004. Hint: a nice way to do this combines the for and the switch structures. 2.20 Develop a well-structured function to determine the elapsed days in a year. The fi rst line of the function should be set up as

function nd = days(mo, da, year)

where mo 5 the month (1–12), da 5 the day (1–31) and year 5 the year. Test it for January 1, 1999; February 29, 2000; March 1, 2001; June 21, 2002; and December 31, 2004. 2.21 Manning’s equation can be used to compute the velocity of water in a rectangular open channel,

U 5 2S

n a BH

B 1 2H b2y3

It is relatively straightforward to compute Cartesian coordinates (x, y) on the basis of polar coordinates (r, u). The reverse process is not so simple. The radius can be computed by the following formula:

r 5 2x2 1 y2

If the coordinates lie within the fi rst and fourth coordinates (i.e., x . 0), then a simple formula can be used to compute u

u 5 tan21 ay x b

The diffi culty arises for the other cases. The following table sum- marizes the possibilities:

x y U

,0 .0 tan21(y/x) 1 p ,0 ,0 tan21(y/x) 2 p ,0 50 p 50 .0 p/2 50 ,0 2p/2 50 50 0

(a) Write a well-structured fl owchart for a subroutine procedure to calculate r and u as a function of x and y. Express the fi nal results for u in degrees.

(b) Write a well-structured function procedure based on your fl owchart. Test your program by using it to fi ll out the follow- ing table:

x y r U

1 0 1 1 0 1 21 1 21 0 21 21 0 21 1 21 0 0

2.15 Develop a well-structured function procedure that is passed a numeric grade from 0 to 100 and returns a letter grade according to the scheme:

cha9792x_ch02_027-054.indd Page 52 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 52 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 53

2.23 The volume V of liquid in a hollow horizontal cylinder of radius r and length L is related to the depth of the liquid h by

V 5 c r 2 cos 21ar 2 h r b 2 (r 2 h)22rh 2 h2 dL

Develop a well-structured function to create a plot of volume versus depth. Test the program for r 5 2 m and L 5 5 m. 2.24 Develop a well-structured program to compute the ve- locity of a parachutist as a function of time using Euler’s method. Test your program for the case where m 5 80 kg and c 5 10 kg/s. Perform the calculation from t 5 0 to 20 s with a step size of 2 s. Use an initial condition that the parachutist has an upward velocity of 20 m/s at t 5 0. At t 5 10 s, assume that the parachute is instantaneously deployed so that the drag coefficient jumps to 50 kg/s. 2.25 The pseudocode in Fig. P2.25 computes the factorial. Express this algorithm as a well-structured function in the language of your choice. Test it by computing 0! and 5!. In addition, test the error trap by trying to evaluate 22!.

FUNCTION fac(n) IF n $ 0 THEN x 5 1 DOFOR i 5 1, n x 5 x ? i END DO fac 5 x ELSE display error message terminate ENDIF END fac

FIGURE P2.25

20.26 The height of a small rocket y can be calculated as a function of time after blastoff with the following piecewise function:

y 5 38.1454t 1 0.13743t3 0 # t , 15

y 5 1036 1 130.909(t 2 15) 1 6.18425(t 2 15)2 2 0.428(t 2 15)3 15 # t , 33

y 5 2900 2 62.468(t 233) 216.9274(t 233)2

1 0.41796(t 233)3 t . 33

where U 5 velocity (m/s), S 5 channel slope, n 5 roughness coef- fi cient, B 5 width (m), and H 5 depth (m). The following data are available for fi ve channels:

n S B H

0.035 0.0001 10 2 0.020 0.0002 8 1 0.015 0.0010 20 1.5 0.030 0.0007 24 3 0.022 0.0003 15 2.5

Write a well-structured program that computes the velocity for each of these channels. Have the program display the input data along with the computed velocity in tabular form where velocity is the fi fth column. Include headings on the table to label the columns. 2.22 A simply supported beam is loaded as shown in Fig. P2.22. Using singularity functions, the displacement along the beam can be expressed by the equation:

uy(x) 5 25

6 [kx 2 0l4 2 kx 2 5l4] 1

15

6 kx 2 8l3

1 75 kx 2 7l2 1 57

6 x3 2 238.25x

By defi nition, the singularity function can be expressed as follows:

kx 2 aln 5 e (x 2 a)n when x . a 0 when x # a

f Develop a program that creates a plot of displacement versus distance along the beam x. Note that x 5 0 at the left end of the beam.

20 kips/ft

150 kip-ft 15 kips

5’ 2’ 1’ 2’

FIGURE P2.22

cha9792x_ch02_027-054.indd Page 53 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 53 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

54 PROGRAMMING AND SOFTWARE

Develop a well-structured pseudocode function to compute y as a function of t. Note that if the user enters a negative value of t or if the rocket has hit the ground (y # 0) then return a value of zero for y. Also, the function should be invoked in the calling program as height(t). Write the algorithm as (a) pseudocode, or (b) in the high-level language of your choice. 20.27 As depicted in Fig. P2.27, a water tank consists of a cylinder topped by the frustum of a cone. Develop a well- structured function in the high-level language or macro lan- guage of your choice to compute the volume given the water level h (m) above the tank’s bottom. Design the function so that it returns a value of zero for negative h’s and the value of the maximum fi lled volume for h’s greater than the tank’s maxi- mum depth. Given the following parameters, H1 5 10 m, r1 5 4 m, H2 5 5 m, and r2 5 6.5 m, test your function by using it to compute the volumes and generate a graph of the volume as a function of level from h 5 21 to 16 m.

h

H2

H1

r1

r2

FIGURE P2.27

cha9792x_ch02_027-054.indd Page 54 09/10/13 7:32 PM F-468 cha9792x_ch02_027-054.indd Page 54 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3 C H A P T E R 3

55

Approximations and Round-Off Errors

Because so many of the methods in this book are straightforward in description and application, it would be very tempting at this point for us to proceed directly to the main body of the text and teach you how to use these techniques. However, understanding the concept of error is so important to the effective use of numerical methods that we have chosen to devote the next two chapters to this topic. The importance of error was introduced in our discussion of the falling parachutist in Chap. 1. Recall that we determined the velocity of a falling parachutist by both ana- lytical and numerical methods. Although the numerical technique yielded estimates that were close to the exact analytical solution, there was a discrepancy, or error, because the numerical method involved an approximation. Actually, we were fortunate in that case because the availability of an analytical solution allowed us to compute the error exactly. For many applied engineering problems, we cannot obtain analytical solutions. Therefore, we cannot compute exactly the errors associated with our numerical methods. In these cases, we must settle for approximations or estimates of the errors. Such errors are characteristic of most of the techniques described in this book. This statement might at fi rst seem contrary to what one normally conceives of as sound engineering. Students and practicing engineers constantly strive to limit errors in their work. When taking examinations or doing homework problems, you are penalized, not rewarded, for your errors. In professional practice, errors can be costly and sometimes catastrophic. If a structure or device fails, lives can be lost. Although perfection is a laudable goal, it is rarely, if ever, attained. For example, despite the fact that the model developed from Newton’s second law is an excellent approximation, it would never in practice exactly predict the parachutist’s fall. A variety of factors such as winds and slight variations in air resistance would result in deviations from the prediction. If these deviations are systematically high or low, then we might need to develop a new model. However, if they are randomly distributed and tightly grouped around the prediction, then the deviations might be considered negligible and the model deemed adequate. Numerical approximations also introduce similar discrepancies into the analysis. Again, the question is: How much the next error is present in our calculations and is it tolerable? This chapter and Chap. 4 cover basic topics related to the identifi cation, quan- tifi cation, and minimization of these errors. In this chapter, general information con- cerned with the quantifi cation of error is reviewed in the fi rst sections. This is

cha9792x_ch03_055-080.indd Page 55 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 55 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

56 APPROXIMATIONS AND ROUND-OFF ERRORS

followed by a section on one of the two major forms of numerical error: round-off error. Round-off error is due to the fact that computers can represent only quantities with a fi nite number of digits. Then Chap. 4 deals with the other major form: trun- cation error. Truncation error is the discrepancy introduced by the fact that numeri- cal methods may employ approximations to represent exact mathematical operations and quantities. Finally, we briefl y discuss errors not directly connected with the numerical methods themselves. These include blunders, formulation or model errors, and data uncertainty.

3.1 SIGNIFICANT FIGURES

This book deals extensively with approximations connected with the manipulation of numbers. Consequently, before discussing the errors associated with numerical methods, it is useful to review basic concepts related to approximate representation of the numbers themselves. Whenever we employ a number in a computation, we must have assurance that it can be used with confi dence. For example, Fig. 3.1 depicts a speedometer and odom- eter from an automobile. Visual inspection of the speedometer indicates that the car is traveling between 48 and 49 km/h. Because the indicator is higher than the midpoint between the markers on the gauge, we can say with assurance that the car is traveling at approximately 49 km/h. We have confi dence in this result because two or more rea- sonable individuals reading this gauge would arrive at the same conclusion. However, let us say that we insist that the speed be estimated to one decimal place. For this case,

40

8 7 3 2 4 4 5

0 120

20

40 60

80

100

FIGURE 3.1 An automobile speedometer and odometer illustrating the concept of a signifi cant fi gure.

cha9792x_ch03_055-080.indd Page 56 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 56 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.1 SIGNIFICANT FIGURES 57

one person might say 48.8, whereas another might say 48.9 km/h. Therefore, because of the limits of this instrument, only the fi rst two digits can be used with confi dence. Estimates of the third digit (or higher) must be viewed as approximations. It would be ludicrous to claim, on the basis of this speedometer, that the automobile is traveling at 48.8642138 km/h. In contrast, the odometer provides up to six certain digits. From Fig. 3.1, we can conclude that the car has traveled slightly less than 87,324.5 km during its lifetime. In this case, the seventh digit (and higher) is uncertain. The concept of a signifi cant fi gure, or digit, has been developed to formally designate the reliability of a numerical value. The signifi cant digits of a number are those that can be used with confi dence. They correspond to the number of certain digits plus one esti- mated digit. For example, the speedometer and the odometer in Fig. 3.1 yield readings of three and seven signifi cant fi gures, respectively. For the speedometer, the two certain digits are 48. It is conventional to set the estimated digit at one-half of the smallest scale division on the measurement device. Thus the speedometer reading would consist of the three signifi cant fi gures: 48.5. In a similar fashion, the odometer would yield a seven- signifi cant-fi gure reading of 87,324.45. Although it is usually a straightforward procedure to ascertain the signifi cant fi gures of a number, some cases can lead to confusion. For example, zeros are not always sig- nifi cant fi gures because they may be necessary just to locate a decimal point. The num- bers 0.00001845, 0.0001845, and 0.001845 all have four signifi cant fi gures. Similarly, when trailing zeros are used in large numbers, it is not clear how many, if any, of the zeros are signifi cant. For example, at face value the number 45,300 may have three, four, or fi ve signifi cant digits, depending on whether the zeros are known with confi dence. Such uncertainty can be resolved by using scientifi c notation, where 4.53 3 104, 4.530 3 104, 4.5300 3 104 designate that the number is known to three, four, and fi ve signifi cant fi gures, respectively. The concept of signifi cant fi gures has two important implications for our study of numerical methods:

1. As introduced in the falling parachutist problem, numerical methods yield approxi- mate results. We must, therefore, develop criteria to specify how confi dent we are in our approximate result. One way to do this is in terms of signifi cant fi gures. For example, we might decide that our approximation is acceptable if it is correct to four signifi cant fi gures.

2. Although quantities such as p, e, or 17 represent specifi c quantities, they cannot be expressed exactly by a limited number of digits. For example,

p 5 3.141592653589793238462643 p

ad infi nitum. Because computers retain only a fi nite number of signifi cant fi gures, such numbers can never be represented exactly. The omission of the remaining signifi cant fi gures is called round-off error.

Both round-off error and the use of signifi cant fi gures to express our confi dence in a numerical result will be explored in detail in subsequent sections. In addition, the concept of signifi cant fi gures will have relevance to our defi nition of accuracy and preci- sion in the next section.

cha9792x_ch03_055-080.indd Page 57 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 57 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

58 APPROXIMATIONS AND ROUND-OFF ERRORS

3.2 ACCURACY AND PRECISION

The errors associated with both calculations and measurements can be characterized with regard to their accuracy and precision. Accuracy refers to how closely a computed or measured value agrees with the true value. Precision refers to how closely individual computed or measured values agree with each other. These concepts can be illustrated graphically using an analogy from target practice. The bullet holes on each target in Fig. 3.2 can be thought of as the predictions of a nu- merical technique, whereas the bull’s-eye represents the truth. Inaccuracy (also called bias) is defi ned as systematic deviation from the truth. Thus, although the shots in Fig. 3.2c are more tightly grouped than those in Fig. 3.2a, the two cases are equally biased because they are both centered on the upper left quadrant of the target. Imprecision (also called uncertainty), on the other hand, refers to the magnitude of the scatter. Therefore, although Fig. 3.2b and d are equally accurate (that is, centered on the bull’s-eye), the latter is more precise because the shots are tightly grouped. Numerical methods should be suffi ciently accurate or unbiased to meet the require- ments of a particular engineering problem. They also should be precise enough for adequate

(c)

(a)

(d)

(b)

Increasing accuracy

In cr

ea si

n g

p re

ci si

o n

FIGURE 3.2 An example from marksmanship illustrating the concepts of accuracy and precision. (a) Inaccurate and imprecise; (b) accurate and imprecise; (c) inaccurate and precise; (d) accurate and precise.

cha9792x_ch03_055-080.indd Page 58 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 58 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.3 ERROR DEFINITIONS 59

engineering design. In this book, we will use the collective term error to represent both the inaccuracy and the imprecision of our predictions. With these concepts as background, we can now discuss the factors that contribute to the error of numerical computations.

3.3 ERROR DEFINITIONS

Numerical errors arise from the use of approximations to represent exact mathematical operations and quantities. These include truncation errors, which result when approxima- tions are used to represent exact mathematical procedures, and round-off errors, which result when numbers having limited signifi cant fi gures are used to represent exact num- bers. For both types, the relationship between the exact, or true, result and the approxi- mation can be formulated as

True value 5 approximation 1 error (3.1)

By rearranging Eq. (3.1), we fi nd that the numerical error is equal to the discrepancy between the truth and the approximation, as in

Et 5 true value 2 approximation (3.2)

where Et is used to designate the exact value of the error. The subscript t is included to designate that this is the “true” error. This is in contrast to other cases, as described shortly, where an “approximate” estimate of the error must be employed. A shortcoming of this defi nition is that it takes no account of the order of magnitude of the value under examination. For example, an error of a centimeter is much more sig- nifi cant if we are measuring a rivet rather than a bridge. One way to account for the mag- nitudes of the quantities being evaluated is to normalize the error to the true value, as in

True fractional relative error 5 true error

true value

where, as specifi ed by Eq. (3.2), error 5 true value 2 approximation. The relative error can also be multiplied by 100 percent to express it as

et 5 true error

true value 100% (3.3)

where et designates the true percent relative error.

EXAMPLE 3.1 Calculation of Errors

Problem Statement. Suppose that you have the task of measuring the lengths of a bridge and a rivet and come up with 9999 and 9 cm, respectively. If the true values are 10,000 and 10 cm, respectively, compute (a) the true error and (b) the true percent rela- tive error for each case.

Solution.

(a) The error for measuring the bridge is [Eq. (3.2)]

Et 5 10,000 2 9999 5 1 cm

cha9792x_ch03_055-080.indd Page 59 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 59 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

60 APPROXIMATIONS AND ROUND-OFF ERRORS

and for the rivet it is

Et 5 10 2 9 5 1 cm

(b) The percent relative error for the bridge is [Eq. (3.3)]

et 5 1

10,000 100% 5 0.01%

and for the rivet it is

et 5 1

10 100% 5 10%

Thus, although both measurements have an error of 1 cm, the relative error for the rivet is much greater. We would conclude that we have done an adequate job of measuring the bridge, whereas our estimate for the rivet leaves something to be desired.

Notice that for Eqs. (3.2) and (3.3), E and e are subscripted with a t to signify that the error is normalized to the true value. In Example 3.1, we were provided with this value. However, in actual situations such information is rarely available. For numerical methods, the true value will be known only when we deal with functions that can be solved analytically. Such will typically be the case when we investigate the theoretical behavior of a particular technique for simple systems. However, in real-world applications, we will obviously not know the true answer a priori. For these situations, an alternative is to normalize the error using the best available estimate of the true value, that is, to the approximation itself, as in

ea 5 approximate error

approximation 100% (3.4)

where the subscript a signifi es that the error is normalized to an approximate value. Note also that for real-world applications, Eq. (3.2) cannot be used to calculate the error term for Eq. (3.4). One of the challenges of numerical methods is to determine error estimates in the absence of knowledge regarding the true value. For example, certain numerical methods use an iterative approach to compute answers. In such an approach, a present approximation is made on the basis of a previous approximation. This process is performed repeatedly, or iteratively, to successively compute (we hope) better and better approxima- tions. For such cases, the error is often estimated as the difference between previous and current approximations. Thus, percent relative error is determined according to

ea 5 current approximation 2 previous approximation

current approximation 100% (3.5)

This and other approaches for expressing errors will be elaborated on in subsequent chapters. The signs of Eqs. (3.2) through (3.5) may be either positive or negative. If the approximation is greater than the true value (or the previous approximation is greater than the current approximation), the error is negative; if the approximation is less than the true value, the error is positive. Also, for Eqs. (3.3) to (3.5), the denominator may

cha9792x_ch03_055-080.indd Page 60 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 60 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.3 ERROR DEFINITIONS 61

be less than zero, which can also lead to a negative error. Often, when performing computations, we may not be concerned with the sign of the error, but we are interested in whether the percent absolute value is lower than a prespecifi ed percent tolerance es. Therefore, it is often useful to employ the absolute value of Eqs. (3.2) through (3.5). For such cases, the computation is repeated until

ZeaZ , es (3.6)

If this relationship holds, our result is assumed to be within the prespecifi ed acceptable level es. Note that for the remainder of this text, we will almost exclusively employ absolute values when we use relative errors. It is also convenient to relate these errors to the number of signifi cant fi gures in the approximation. It can be shown (Scarborough, 1966) that if the following criterion is met, we can be assured that the result is correct to at least n signifi cant fi gures.

es 5 (0.5 3 1022n)% (3.7)

EXAMPLE 3.2 Error Estimates for Iterative Methods

Problem Statement. In mathematics, functions can often be represented by infi nite series. For example, the exponential function can be computed using

ex 5 1 1 x 1 x 2

2 1

x 3

3! 1 p 1

x  n

n! (E3.2.1)

Thus, as more terms are added in sequence, the approximation becomes a better and better estimate of the true value of ex. Equation (E3.2.1) is called a Maclaurin series expansion. Starting with the simplest version, ex 5 1, add terms one at a time to estimate e0.5. After each new term is added, compute the true and approximate percent relative errors with Eqs. (3.3) and (3.5), respectively. Note that the true value is e0.5 5 1.648721 . . . . Add terms until the absolute value of the approximate error estimate ea falls below a prespecifi ed error criterion es conforming to three signifi cant fi gures.

Solution. First, Eq. (3.7) can be employed to determine the error criterion that ensures a result is correct to at least three signifi cant fi gures:

es 5 (0.5 3 10223)% 5 0.05%

Thus, we will add terms to the series until ea falls below this level. The fi rst estimate is simply equal to Eq. (E3.2.1) with a single term. Thus, the fi rst es- timate is equal to 1. The second estimate is then generated by adding the second term, as in

ex 5 1 1 x

or for x 5 0.5,

e0.5 5 1 1 0.5 5 1.5

This represents a true percent relative error of [Eq. (3.3)]

et 5 1.648721 2 1.5

1.648721 100% 5 9.02%

cha9792x_ch03_055-080.indd Page 61 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 61 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

62 APPROXIMATIONS AND ROUND-OFF ERRORS

Equation (3.5) can be used to determine an approximate estimate of the error, as in

ea 5 1.5 2 1

1.5 100% 5 33.3%

Because ea is not less than the required value of es, we would continue the computation by adding another term, x2y2!, and repeating the error calculations. The process is con- tinued until ea , es. The entire computation can be summarized as

Terms Result Et (%) Ea (%)

1 1 39.3 2 1.5 9.02 33.3 3 1.625 1.44 7.69 4 1.645833333 0.175 1.27 5 1.648437500 0.0172 0.158 6 1.648697917 0.00142 0.0158

Thus, after six terms are included, the approximate error falls below es 5 0.05% and the computation is terminated. However, notice that, rather than three signifi cant fi gures, the result is accurate to fi ve! This is because, for this case, both Eqs. (3.5) and (3.7) are con- servative. That is, they ensure that the result is at least as good as they specify. Although, as discussed in Chap. 6, this is not always the case for Eq. (3.5), it is true most of the time.

3.3.1 Computer Algorithm for Iterative Calculations

Many of the numerical methods described in the remainder of this text involve iterative cal- culations of the sort illustrated in Example 3.2. These all entail solving a mathematical problem by computing successive approximations to the solution starting from an initial guess. The computer implementation of such iterative solutions involves loops. As we saw in Sec. 2.1.1, these come in two basic fl avors: count-controlled and decision loops. Most iterative solutions use decision loops. Thus, rather than employing a prespecifi ed number of iterations, the process typically is repeated until an approximate error estimate falls below a stopping criterion, as in Example 3.2. A pseudocode for a generic iterative calculation is presented in Fig. 3.3. The function is passed a value (val) along with a stopping error criterion (es) and a maximum al- lowable number of iterations (maxit). The value is typically either (1) an initial value or (2) the value for which the iterative calculation is to be made. The function fi rst initializes three variables. These include (1) a variable iter that keeps track of the number of iterations, (2) a variable sol that holds the current estimate of the solution, and (3) a variable ea that holds the approximate percent relative error. Note that ea is initially set to a value of 100 to ensure that the loop executes at least once. These initializations are followed by the decision loop that actually implements the iterative calculation. Prior to generating a new solution, sol is fi rst assigned to solold. Then a new value of sol is computed and the iteration counter is incremented. If the new value of sol is nonzero, the percent relative error ea is determined. The stopping

cha9792x_ch03_055-080.indd Page 62 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 62 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.3 ERROR DEFINITIONS 63

criteria are then tested. If both are false, the loop repeats. If either are true, the loop terminates and the fi nal solution is sent back to the function call. The following example illustrates how the generic algorithm can be applied to a specifi c iterative calculation.

EXAMPLE 3.3 Computer Implementation of an Iterative Calculation

Problem Statement. Develop a computer program based on the pseudocode from Fig. 3.3 to implement the calculation from Example 3.2.

Solution. A function to implement the Maclaurin series expansion for ex can be based on the general scheme in Fig. 3.3. To do this, we fi rst formulate the series expansion as a formula:

ex >a n

i50

xn

n!

Figure 3.4 shows functions to implement this series written in VBA and MATLAB software. Similar codes could be developed in other languages such a C11 or Fortran 95. Notice that whereas MATLAB has a built-in factorial function, it is necessary to compute the factorial as part of the VBA implementation with a simple product accumulator fac. When the programs are run, they generate an estimate for the exponential function. For the MATLAB version, the answer is returned along with the approximate error and the number of iterations. For example, e1 can be evaluated as

>> format long >> [val, ea, iter] = IterMeth(1,1e−6,100)

val = 2.718281826198493 ea = 9.216155641522974e−007 iter = 12

FUNCTION IterMeth(val, es, maxit) iter 5 1 sol 5 val ea 5 100 DO solold 5 sol sol 5 ... iter 5 iter 1 1 IF sol fi 0 ea5abs((sol 2 solold)/sol)*100 IF ea # es OR iter $ maxit EXIT END DO IterMeth 5 sol END IterMeth

FIGURE 3.3 Pseudocode for a generic iterative calculation.

cha9792x_ch03_055-080.indd Page 63 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 63 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

64 APPROXIMATIONS AND ROUND-OFF ERRORS

We can see that after 12 iterations, we obtain a result of 2.7182818 with an approxi- mate error estimate of 5 9.2162 3 1027%. The result can be verifi ed by using the built-in exp function to directly calculate the exact value and the true percent relative error,

>> trueval=exp(1)

trueval = 2.718281828459046

>> et=abs((trueval−val)/trueval)*100 et = 8.316108397236229e−008

As was the case with Example 3.2, we obtain the desirable outcome that the true error is less than the approximate error.

With the preceding defi nitions as background, we can now proceed to the two types of error connected directly with numerical methods: round-off errors and truncation errors.

(b) MATLAB function [v,ea,iter] = IterMeth(x,es,maxit) % initialization iter = 1; sol = 1; ea = 100;

% iterative calculation while (1) solold = sol;

sol = sol + x ^ iter / factorial(iter); iter = iter + 1; if sol~=0 ea=abs((sol − solold)/sol)*100; end if ea<=es | iter>=maxit,break,end end v = sol; end

(a) VBA/Excel Function IterMeth(x, es, maxit) ’ initialization iter = 1 sol = 1 ea = 100 fac = 1 ’ iterative calculation Do solold = sol fac = fac * iter sol = sol + x ^ iter / fac iter = iter + 1 If sol <> 0 Then ea = Abs((sol − solold) / sol) * 100 End If If ea <= es Or iter >= maxit Then Exit Do Loop IterMeth = sol End Function

FIGURE 3.4 (a) VBA/Excel and (b) MATLAB functions based on the pseudocode from Fig. 3.3.

cha9792x_ch03_055-080.indd Page 64 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 64 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 65

3.4 ROUND-OFF ERRORS

As mentioned previously, round-off errors originate from the fact that computers retain only a fi xed number of signifi cant fi gures during a calculation. Numbers such as p, e, or 27 cannot be expressed by a fi xed number of signifi cant fi gures. Therefore, they cannot be represented exactly by the computer. In addition, because computers use a base-2 representation, they cannot precisely represent certain exact base-10 numbers. The discrepancy introduced by this omission of signifi cant fi gures is called round-off error.

3.4.1 Computer Representation of Numbers

Numerical round-off errors are directly related to the manner in which numbers are stored in a computer. The fundamental unit whereby information is represented is called a word. This is an entity that consists of a string of binary digits, or bits. Numbers are typically stored in one or more words. To understand how this is accomplished, we must fi rst review some material related to number systems.

Number Systems. A number system is merely a convention for representing quantities. Because we have 10 fi ngers and 10 toes, the number system that we are most familiar with is the decimal, or base-10, number system. A base is the number used as the refer- ence for constructing the system. The base-10 system uses the 10 digits—0, 1, 2, 3, 4, 5, 6, 7, 8, 9—to represent numbers. By themselves, these digits are satisfactory for counting from 0 to 9. For larger quantities, combinations of these basic digits are used, with the position or place value specifying the magnitude. The right-most digit in a whole number repre- sents a number from 0 to 9. The second digit from the right represents a multiple of 10. The third digit from the right represents a multiple of 100 and so on. For example, if we have the number 86,409 then we have eight groups of 10,000, six groups of 1000, four groups of 100, zero groups of 10, and nine more units, or

(8 3 104) 1 (6 3 103) 1 (4 3 102) 1 (0 3 101) 1 (9 3 100) 5 86,409

Figure 3.5a provides a visual representation of how a number is formulated in the base-10 system. This type of representation is called positional notation. Because the decimal system is so familiar, it is not commonly realized that there are alternatives. For example, if human beings happened to have had eight fi ngers and eight toes, we would undoubtedly have developed an octal, or base-8, representation. In the same sense, our friend the computer is like a two-fi ngered animal who is limited to two states—either 0 or 1. This relates to the fact that the primary logic units of digital com- puters are on/off electronic components. Hence, numbers on the computer are represented with a binary, or base-2, system. Just as with the decimal system, quantities can be represented using positional notation. For example, the binary number 11 is equivalent to (1 3 21) 1 (1 3 20) 5 2 1 1 5 3 in the decimal system. Figure 3.5b illustrates a more complicated example.

Integer Representation. Now that we have reviewed how base-10 numbers can be represented in binary form, it is simple to conceive of how integers are represented on a computer. The most straightforward approach, called the signed magnitude method, employs the fi rst bit of a word to indicate the sign, with a 0 for positive and a 1 for

cha9792x_ch03_055-080.indd Page 65 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 65 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

66 APPROXIMATIONS AND ROUND-OFF ERRORS

negative. The remaining bits are used to store the number. For example, the integer value of 2173 would be stored on a 16-bit computer, as in Fig. 3.6.

EXAMPLE 3.4 Range of Integers

Problem Statement. Determine the range of integers in base-10 that can be represented on a 16-bit computer.

FIGURE 3.5 How the (a) decimal (base-10) and the (b) binary (base-2) systems work. In (b), the binary num- ber 10101101 is equivalent to the decimal number 173.

1 � 1 = 0 � 2 = 1 � 4 = 1 � 8 = 0 � 16 = 1 � 32 = 0 � 64 = 1 � 128 =

1 0 4 8 0

32 0

128 173

27

1

26

0

25

1

24

0

23

1

22

1

21

0

20

1

9 � 1 = 0 � 10 = 4 � 100 = 6 � 1,000 = 8 � 10,000 =

9 0

400 6,000

80,000 86,409

104

8

103

6

102

4

101

0

100

9

(a)

(b)

FIGURE 3.6 The representation of the decimal integer 2173 on a 16-bit computer using the signed magnitude method.

1 0 0 0 0 0 0 0 1 0 1 0 1 1 0 1

Sign Number

cha9792x_ch03_055-080.indd Page 66 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 66 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 67

Solution. Of the 16 bits, the fi rst bit holds the sign. The remaining 15 bits can hold binary numbers from 0 to 111111111111111. The upper limit can be converted to a decimal integer, as in

(1 3 214) 1 (1 3 213) 1 p 1 (1 3 21) 1 (1 3 20)

which equals 32,767 (note that this expression can be simply evaluated as 215 2 1). Thus, a 16-bit computer word can store decimal integers ranging from 232,767 to 32,767. In addition, because zero is already defi ned as 0000000000000000, it is redundant to use the number 1000000000000000 to defi ne a “minus zero.” Therefore, it is usually em- ployed to represent an additional negative number: 232,768, and the range is from 232,768 to 32,767.

Note that the signed magnitude method described above is not used to represent integers on conventional computers. A preferred approach called the 2’s complement technique directly incorporates the sign into the number’s magnitude rather than provid- ing a separate bit to represent plus or minus (see Chapra and Canale 1994). However, Example 3.4 still serves to illustrate how all digital computers are limited in their capa- bility to represent integers. That is, numbers above or below the range cannot be repre- sented. A more serious limitation is encountered in the storage and manipulation of fractional quantities as described next.

Floating-Point Representation. Fractional quantities are typically represented in com- puters using fl oating-point form. In this approach, the number is expressed as a fractional part, called a mantissa or signifi cand, and an integer part, called an exponent or charac- teristic, as in

m # be

where m 5 the mantissa, b 5 the base of the number system being used, and e 5 the exponent. For instance, the number 156.78 could be represented as 0.15678 3 103 in a fl oating-point base-10 system. Figure 3.7 shows one way that a fl oating-point number could be stored in a word. The fi rst bit is reserved for the sign, the next series of bits for the signed exponent, and the last bits for the mantissa.

FIGURE 3.7 The manner in which a fl oating-point number is stored in a word.

Sign

Signed exponent

Mantissa

cha9792x_ch03_055-080.indd Page 67 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 67 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

68 APPROXIMATIONS AND ROUND-OFF ERRORS

Note that the mantissa is usually normalized if it has leading zero digits. For ex- ample, suppose the quantity 1y34 5 0.029411765 . . . was stored in a fl oating-point base- 10 system that allowed only four decimal places to be stored. Thus, 1y34 would be stored as

0.0294 3 100

However, in the process of doing this, the inclusion of the useless zero to the right of the decimal forces us to drop the digit 1 in the fi fth decimal place. The number can be normalized to remove the leading zero by multiplying the mantissa by 10 and lowering the exponent by 1 to give

0.2941 3 1021

Thus, we retain an additional signifi cant fi gure when the number is stored. The consequence of normalization is that the absolute value of m is limited. That is,

1

b # m , 1 (3.8)

where b 5 the base. For example, for a base-10 system, m would range between 0.1 and 1, and for a base-2 system, between 0.5 and 1. Floating-point representation allows both fractions and very large numbers to be expressed on the computer. However, it has some disadvantages. For example, floating-point numbers take up more room and take longer to process than integer numbers. More significantly, however, their use introduces a source of error because the mantissa holds only a finite number of significant figures. Thus, a round-off error is introduced.

EXAMPLE 3.5 Hypothetical Set of Floating-Point Numbers

Problem Statement. Create a hypothetical fl oating-point number set for a machine that stores information using 7-bit words. Employ the fi rst bit for the sign of the number, the next three for the sign and the magnitude of the exponent, and the last three for the magnitude of the mantissa (Fig. 3.8).

FIGURE 3.8 The smallest possible positive fl oating-point number from Example 3.5.

0 1 1 1 1 0 0

Sign of number

Sign of exponent

Magnitude of exponent

Magnitude of mantissa

21 20 2–1 2–2 2–3

cha9792x_ch03_055-080.indd Page 68 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 68 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 69

Solution. The smallest possible positive number is depicted in Fig. 3.8. The initial 0 indicates that the quantity is positive. The 1 in the second place designates that the exponent has a negative sign. The 1’s in the third and fourth places give a maximum value to the exponent of

1 3 21 1 1 3 20 5 3

Therefore, the exponent will be 23. Finally, the mantissa is specifi ed by the 100 in the last three places, which conforms to

1 3 221 1 0 3 222 1 0 3 223 5 0.5

Although a smaller mantissa is possible (e.g., 000, 001, 010, 011), the value of 100 is used because of the limit imposed by normalization [Eq. (3.8)]. Thus, the smallest possible positive number for this system is 10.5 3 223, which is equal to 0.0625 in the base-10 system. The next highest numbers are developed by increasing the mantissa, as in

0111101 5 (1 3 221 1 0 3 222 1 1 3 223) 3 223 5 (0.078125)10 0111110 5 (1 3 221 1 1 3 222 1 0 3 223) 3 223 5 (0.093750)10 0111111 5 (1 3 221 1 1 3 222 1 1 3 223) 3 223 5 (0.109375)10

Notice that the base-10 equivalents are spaced evenly with an interval of 0.015625. At this point, to continue increasing, we must decrease the exponent to 10, which gives a value of

1 3 21 1 0 3 20 5 2

The mantissa is decreased back to its smallest value of 100. Therefore, the next num- ber is

0110100 5 (1 3 221 1 0 3 222 1 0 3 223) 3 222 5 (0.125000)10 This still represents a gap of 0.125000 2 0.109375 5 0.015625. However, now when higher numbers are generated by increasing the mantissa, the gap is lengthened to 0.03125,

0110101 5 (1 3 221 1 0 3 222 1 1 3 223) 3 222 5 (0.156250)10 0110110 5 (1 3 221 1 1 3 222 1 0 3 223) 3 222 5 (0.187500)10 0110111 5 (1 3 221 1 1 3 222 1 1 3 223) 3 222 5 (0.218750)10

This pattern is repeated as each larger quantity is formulated until a maximum number is reached,

0011111 5 (1 3 221 1 1 3 222 1 1 3 223) 3 23 5 (7)10 The fi nal number set is depicted graphically in Fig. 3.9.

Figure 3.9 manifests several aspects of fl oating-point representation that have signifi cance regarding computer round-off errors:

1. There Is a Limited Range of Quantities That May Be Represented. Just as for the integer case, there are large positive and negative numbers that cannot be represented. Attempts to employ numbers outside the acceptable range will result in what is called

cha9792x_ch03_055-080.indd Page 69 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 69 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

70 APPROXIMATIONS AND ROUND-OFF ERRORS

an overfl ow error. However, in addition to large quantities, the fl oating-point repre- sentation has the added limitation that very small numbers cannot be represented. This is illustrated by the underfl ow “hole” between zero and the fi rst positive number in Fig. 3.9. It should be noted that this hole is enlarged because of the normalization constraint of Eq. (3.8).

2. There Are Only a Finite Number of Quantities That Can Be Represented within the Range. Thus, the degree of precision is limited. Obviously, irrational numbers cannot be represented exactly. Furthermore, rational numbers that do not exactly match one of the values in the set also cannot be represented precisely. The errors introduced by approximating both these cases are referred to as quantizing errors. The actual approximation is accomplished in either of two ways: chopping or rounding. For example, suppose that the value of p 5 3.14159265358 . . . is to be stored on a base- 10 number system carrying seven signifi cant fi gures. One method of approximation would be to merely omit, or “chop off,” the eighth and higher terms, as in p 5 3.141592, with the introduction of an associated error of [Eq. (3.2)]

Et 5 0.00000065 p

This technique of retaining only the signifi cant terms was originally dubbed “truncation” in computer jargon. We prefer to call it chopping to distinguish it from the truncation errors discussed in Chap. 4. Note that for the base-2 number system

�x x – �x

�x/2 �x/2 x – �x x + �x

Chopping Rounding

0

0

7 Overflow

Underflow “hole” at zero

FIGURE 3.9 The hypothetical number system developed in Example 3.5. Each value is indicated by a tick mark. Only the positive numbers are shown. An identical set would also extend in the negative direction.

cha9792x_ch03_055-080.indd Page 70 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 70 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 71

in Fig. 3.9, chopping means that any quantity falling within an interval of length Dx will be stored as the quantity at the lower end of the interval. Thus, the upper error bound for chopping is Dx. Additionally, a bias is introduced because all errors are positive. The shortcomings of chopping are attributable to the fact that the higher terms in the complete decimal representation have no impact on the shortened version. For instance, in our example of p, the fi rst discarded digit is 6. Thus, the last retained digit should be rounded up to yield 3.141593. Such rounding reduces the error to

Et 5 20.00000035 p

Consequently, rounding yields a lower absolute error than chopping. Note that for the base-2 number system in Fig. 3.9, rounding means that any quantity falling within an interval of length Dx will be represented as the nearest allowable number. Thus, the upper error bound for rounding is Dxy2. Additionally, no bias is introduced because some errors are positive and some are negative. Some computers employ rounding. However, this adds to the computational overhead, and, consequently, many machines use simple chopping. This approach is justifi ed under the supposition that the number of signifi cant fi gures is large enough that resulting round-off error is usually negligible.

3. The Interval between Numbers, Dx, Increases as the Numbers Grow in Magnitude. It is this characteristic, of course, that allows fl oating-point representation to preserve signifi cant digits. However, it also means that quantizing errors will be proportional to the magnitude of the number being represented. For normalized fl oating-point numbers, this proportionality can be expressed, for cases where chopping is employed, as

Z¢x Z Z x Z

# e (3.9)

and, for cases where rounding is employed, as

Z¢x Z Z x Z

# e

2 (3.10)

where % is referred to as the machine epsilon, which can be computed as

e 5 b12 t (3.11)

where b is the number base and t is the number of signifi cant digits in the mantissa. Notice that the inequalities in Eqs. (3.9) and (3.10) signify that these are error bounds. That is, they specify the worst cases.

EXAMPLE 3.6 Machine Epsilon

Problem Statement. Determine the machine epsilon and verify its effectiveness in char- acterizing the errors of the number system from Example 3.5. Assume that chopping is used.

Solution. The hypothetical fl oating-point system from Example 3.5 employed values of the base b 5 2, and the number of mantissa bits t 5 3. Therefore, the machine epsi- lon would be [Eq. (3.11)]

e 5 2123 5 0.25

cha9792x_ch03_055-080.indd Page 71 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 71 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

72 APPROXIMATIONS AND ROUND-OFF ERRORS

Consequently, the relative quantizing error should be bounded by 0.25 for chopping. The largest relative errors should occur for those quantities that fall just below the upper bound of the fi rst interval between successive equispaced numbers (Fig. 3.10). Those numbers falling in the succeeding higher intervals would have the same value of Dx but a greater value of x and, hence, would have a lower relative error. An example of a maximum error would be a value falling just below the upper bound of the interval between (0.125000)10 and (0.156250)10. For this case, the error would be less than

0.03125

0.125000 5 0.25

Thus, the error is as predicted by Eq. (3.9).

Largest relative error

FIGURE 3.10 The largest quantizing error will occur for those values falling just below the upper bound of the fi rst of a series of equispaced intervals.

The magnitude dependence of quantizing errors has a number of practical applica- tions in numerical methods. Most of these relate to the commonly employed operation of testing whether two numbers are equal. This occurs when testing convergence of quantities as well as in the stopping mechanism for iterative processes (recall Example 3.2). For these cases, it should be clear that, rather than test whether the two quantities are equal, it is advisable to test whether their difference is less than an acceptably small tolerance. Further, it should also be evident that normalized rather than absolute differ- ence should be compared, particularly when dealing with numbers of large magnitude. In addition, the machine epsilon can be employed in formulating stopping or convergence criteria. This ensures that programs are portable—that is, they are not dependent on the computer on which they are implemented. Figure 3.11 lists pseudocode to automatically determine the machine epsilon of a binary computer.

Extended Precision. It should be noted at this point that, although round-off errors can be important in contexts such as testing convergence, the number of signifi cant digits carried on most computers allows most engineering computations to be performed with more than acceptable precision. For example, the hypothetical number system in Fig. 3.9 is a gross exaggeration that was employed for illustrative purposes. Commercial computers use much larger words and, consequently, allow numbers to be expressed with more than adequate precision. For example, computers that use IEEE format allow 24 bits to be used for the mantissa, which translates into about seven signifi cant base-10 digits of precision1 with a range of about 10238 to 1039.

FIGURE 3.11 Pseudocode to determine machine epsilon for a binary computer.

epsilon 5 1 DO IF (epsilon11 # 1)EXIT epsilon 5 epsilon/2 END DO epsilon 5 2 3 epsilon

1Note that only 23 bits are actually used to store the mantissa. However, because of normalization, the fi rst bit of the mantissa is always 1 and is, therefore, not stored. Thus, this fi rst bit together with the 23 stored bits gives the 24 total bits of precision for the mantissa.

cha9792x_ch03_055-080.indd Page 72 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 72 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 73

With this acknowledged, there are still cases where round-off error becomes critical. For this reason most computers allow the specifi cation of extended precision. The most common of these is double precision, in which the number of words used to store fl oating-point numbers is doubled. It provides about 15 to 16 decimal digits of precision and a range of approximately 102308 to 10308. In many cases, the use of double-precision quantities can greatly mitigate the effect of round-off errors. However, a price is paid for such remedies in that they also require more memory and execution time. The difference in execution time for a small calcula- tion might seem insignifi cant. However, as your programs become larger and more com- plicated, the added execution time could become considerable and have a negative impact on your effectiveness as a problem solver. Therefore, extended precision should not be used frivolously. Rather, it should be selectively employed where it will yield the maxi- mum benefi t at the least cost in terms of execution time. In the following sections, we will look closer at how round-off errors affect computations, and in so doing provide a foundation of understanding to guide your use of the double-precision capability. Before proceeding, it should be noted that some of the commonly used software pack- ages (for example, Excel, Mathcad) routinely use double precision to represent numerical quantities. Thus, the developers of these packages decided that mitigating round-off errors would take precedence over any loss of speed incurred by using extended precision. Others, like MATLAB software, allow you to use extended precision, if you desire.

3.4.2 Arithmetic Manipulations of Computer Numbers

Aside from the limitations of a computer’s number system, the actual arithmetic manipula- tions involving these numbers can also result in round-off error. In the following section, we will fi rst illustrate how common arithmetic operations affect round-off errors. Then we will investigate a number of particular manipulations that are especially prone to round-off errors.

Common Arithmetic Operations. Because of their familiarity, normalized base-10 numbers will be employed to illustrate the effect of round-off errors on simple addition, subtraction, multiplication, and division. Other number bases would behave in a similar fashion. To simplify the discussion, we will employ a hypothetical decimal computer with a 4-digit mantissa and a 1-digit exponent. In addition, chopping is used. Rounding would lead to similar though less dramatic errors. When two fl oating-point numbers are added, the mantissa of the number with the smaller exponent is modifi ed so that the exponents are the same. This has the effect of align- ing the decimal points. For example, suppose we want to add 0.1557 ? 101 1 0.4381 ? 1021. The decimal of the mantissa of the second number is shifted to the left a number of places equal to the difference of the exponents [1 2 (21) 5 2], as in

0.4381 # 1021S 0.004381 # 101

Now the numbers can be added,

0.1557 # 101

0.004381 # 101

0.160081 # 101

and the result chopped to 0.1600 ? 101. Notice how the last two digits of the second number that were shifted to the right have essentially been lost from the computation.

cha9792x_ch03_055-080.indd Page 73 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 73 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

74 APPROXIMATIONS AND ROUND-OFF ERRORS

Subtraction is performed identically to addition except that the sign of the subtrahend is reversed. For example, suppose that we are subtracting 26.86 from 36.41. That is,

0.3641 # 102

20.2686 # 102

0.0955 # 102

For this case the result is not normalized, and so we must shift the decimal one place to the right to give 0.9550 ? 101 5 9.550. Notice that the zero added to the end of the man- tissa is not signifi cant but is merely appended to fi ll the empty space created by the shift. Even more dramatic results would be obtained when the numbers are very close, as in

0.7642 # 103

20.7641 # 103

0.0001 # 103

which would be converted to 0.1000 ? 100 5 0.1000. Thus, for this case, three nonsig- nifi cant zeros are appended. This introduces a substantial computational error because subsequent manipulations would act as if these zeros were signifi cant. As we will see in a later section, the loss of signifi cance during the subtraction of nearly equal numbers is among the greatest source of round-off error in numerical methods. Multiplication and division are somewhat more straightforward than addition or sub- traction. The exponents are added and the mantissas multiplied. Because multiplication of two n-digit mantissas will yield a 2n-digit result, most computers hold intermediate results in a double-length register. For example,

0.1363 # 103 3 0.6423 # 1021 5 0.08754549 # 102

If, as in this case, a leading zero is introduced, the result is normalized,

0.08754549 # 102S 0.8754549 # 101

and chopped to give

0.8754 # 101

Division is performed in a similar manner, but the mantissas are divided and the exponents are subtracted. Then the results are normalized and chopped.

Large Computations. Certain methods require extremely large numbers of arithmetic manipulations to arrive at their fi nal results. In addition, these computations are often interdependent. That is, the later calculations are dependent on the results of earlier ones. Consequently, even though an individual round-off error could be small, the cumulative effect over the course of a large computation can be signifi cant.

EXAMPLE 3.7 Large Numbers of Interdependent Computations

Problem Statement. Investigate the effect of round-off error on large numbers of in- terdependent computations. Develop a program to sum a number 100,000 times. Sum the number 1 in single precision, and 0.00001 in single and double precision.

Solution. Figure 3.12 shows a Fortran 90 program that performs the summation. Whereas the single-precision summation of 1 yields the expected result, the single-precision

cha9792x_ch03_055-080.indd Page 74 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 74 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 75

summation of 0.00001 yields a large discrepancy. This error is reduced signifi cantly when 0.00001 is summed in double precision. Quantizing errors are the source of the discrepancies. Because the integer 1 can be represented exactly within the computer, it can be summed exactly. In contrast, 0.00001 cannot be represented exactly and is quantized by a value that is slightly different from its true value. Whereas this very slight discrepancy would be negligible for a small com- putation, it accumulates after repeated summations. The problem still occurs in double precision but is greatly mitigated because the quantizing error is much smaller.

PROGRAM fig0312 IMPLICIT none INTEGER::i REAL::sum1, sum2, x1, x2 DOUBLE PRECISION::sum3, x3 sum1=0. sum2=0. sum3=0. x1=1. x2=1.e−5 x3=1.d−5 DO i=1,100000 sum1=sum1+x1 sum2=sum2+x2 sum3=sum3+x3 END DO PRINT *, sum1 PRINT *, sum2 PRINT *, sum3 END output: 100000.000000 1.000990 9.999999999980838E-001

FIGURE 3.12 Fortran 90 program to sum a number 105 times. The case sums the number 1 in single precision and the number 1025 in single and double precision.

Note that the type of error illustrated by the previous example is somewhat atypical in that all the errors in the repeated operation are of the same sign. In most cases the errors of a long computation alternate sign in a random fashion and, thus, often cancel out. However, there are also instances where such errors do not cancel but, in fact, lead to a spurious fi nal result. The following sections are intended to provide insight into ways in which this may occur.

Adding a Large and a Small Number. Suppose we add a small number, 0.0010, to a large number, 4000, using a hypothetical computer with the 4-digit mantissa and the 1-digit exponent. We modify the smaller number so that its exponent matches the larger,

0.4000 # 104

0.0000001 # 104

0.4000001 # 104

cha9792x_ch03_055-080.indd Page 75 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 75 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

76 APPROXIMATIONS AND ROUND-OFF ERRORS

which is chopped to 0.4000 ? 104. Thus, we might as well have not performed the addition! This type of error can occur in the computation of an infi nite series. The initial terms in such series are often relatively large in comparison with the later terms. Thus, after a few terms have been added, we are in the situation of adding a small quantity to a large quantity. One way to mitigate this type of error is to sum the series in reverse order—that is, in ascending rather than descending order. In this way, each new term will be of com- parable magnitude to the accumulated sum (see Prob. 3.5).

Subtractive Cancellation. This term refers to the round-off induced when subtracting two nearly equal fl oating-point numbers. One common instance where this can occur involves fi nding the roots of a quadratic equation or parabola with the quadratic formula,

x1 x2

5 2b62b224ac

2a (3.12)

For cases where b2 W 4ac, the difference in the numerator can be very small. In such cases, double precision can mitigate the problem. In addition, an alternative formulation can be used to minimize subtractive cancellation,

x1 x2

5 22c

b 6 2b2 2 4ac (3.13)

An illustration of the problem and the use of this alternative formula are provided in the following example.

EXAMPLE 3.8 Subtractive Cancellation

Problem Statement. Compute the values of the roots of a quadratic equation with a 5 1, b 5 3000.001, and c 5 3. Check the computed values versus the true roots of x1 5 20.001 and x2 5 23000.

Solution. Figure 3.13 shows an Excel/VBA program that computes the roots x1 and x2 on the basis of the quadratic formula [(Eq. (3.12)]. Note that both single- and double-precision versions are given. Whereas the results for x2 are adequate, the percent relative errors for x1 are poor for the single-precision version, et 5 2.4%. This level could be inadequate for many applied engineering problems. This result is particularly surprising because we are employing an analytical formula to obtain our solution! The loss of signifi cance occurs in the line of both programs where two relatively large numbers are subtracted. Similar problems do not occur when the same numbers are added. On the basis of the above, we can draw the general conclusion that the quadratic formula will be susceptible to subtractive cancellation whenever b2 W 4ac. One way to circumvent this problem is to use double precision. Another is to recast the quadratic formula in the format of Eq. (3.13). As in the program output, both options give a much smaller error because the subtractive cancellation is minimized or avoided.

cha9792x_ch03_055-080.indd Page 76 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 76 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

3.4 ROUND-OFF ERRORS 77

Option Explicit

Sub fig0313() Dim a As Single, b As Single Dim c As Single, d As Single Dim x1 As Single, x2 As Single Dim x1r As Single Dim aa As Double, bb As Double Dim cc As Double, dd As Double Dim x11 As Double, x22 As Double

'Single precision: a = 1: b = 3000.001: c = 3 d = Sqr(b * b − 4 * a * c) x1 = (−b + d) / (2 * a) x2 = (−b − d) / (2 * a)

'Double precision: aa = 1: bb = 3000.001: cc = 3 dd = Sqr(bb * bb − 4 * aa * cc) x11 = (−bb + dd) / (2 * aa) x22 = (−bb − dd) / (2 * aa)

'Modified formula for first root 'single precision: x1r = −2 * c / (b + d)

FIGURE 3.13 Excel/VBA program to determine the roots of a quadratic.

'Display results Sheets("sheet1").Select Range("b2").Select ActiveCell.Value = x1 ActiveCell.Offset(1, 0).Select ActiveCell.Value = x2 ActiveCell.Offset(2, 0).Select ActiveCell.Value = x11 ActiveCell.Offset(1, 0).Select ActiveCell.Value = x22 ActiveCell.Offset(2, 0).Select ActiveCell.Value = x1r End Sub

OUTPUT:

Note that, as in the foregoing example, there are times when subtractive cancellation can be circumvented by using a transformation. However, the only general remedy is to employ extended precision.

Smearing. Smearing occurs whenever the individual terms in a summation are larger than the summation itself. As in the following example, one case where this occurs is in series of mixed signs.

EXAMPLE 3.9 Evaluation of ex using Infi nite Series

Problem Statement. The exponential function y 5 ex is given by the infi nite series

y 5 1 1 x 1 x2

2 1

x3

3! 1 p

Evaluate this function for x 5 10 and x 5 210, and be attentive to the problems of round-off error.

Solution. Figure 3.14a gives an Excel/VBA program that uses the infi nite series to evaluate ex. The variable i is the number of terms in the series, term is the value of the

cha9792x_ch03_055-080.indd Page 77 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 77 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

78 APPROXIMATIONS AND ROUND-OFF ERRORS

current term added to the series, and sum is the accumulative value of the series. The variable test is the preceding accumulative value of the series prior to adding term. The series is terminated when the computer cannot detect the difference between test and sum. Figure 3.14b shows the results of running the program for x 5 10. Note that this case is completely satisfactory. The fi nal result is achieved in 31 terms with the series identical to the library function value within seven signifi cant fi gures. Figure 3.14c shows similar results for x 5 210. However, for this case, the results of the series calculation are not even the same sign as the true result. As a matter of fact, the negative results are open to serious question because ex can never be less than zero. The problem here is caused by round-off error. Note that many of the terms that make up the

(a) Program

Option Explicit

Sub fig0314() Dim term As Single, test As Single Dim sum As Single, x As Single Dim i As Integer i = 0: term = 1#: sum = 1#: test = 0# Sheets("sheet1").Select Range("b1").Select x = ActiveCell.Value Range("a3:c1003").ClearContents Range("a3").Select Do If sum = test Then Exit Do ActiveCell.Value = i ActiveCell.Offset(0, 1).Select ActiveCell.Value = term ActiveCell.Offset(0, 1).Select ActiveCell.Value = sum ActiveCell.Offset(1, -2).Select i = i + 1 test = sum term = x ^ i / _ Application.WorksheetFunction.Fact(i) sum = sum + term Loop ActiveCell.Offset(0, 1).Select ActiveCell.Value = "Exact value = " ActiveCell.Offset(0, 1).Select ActiveCell.Value = Exp(x) End Sub

(b) Evaluation of e10

(c) Evaluation of e10

FIGURE 3.14 (a) An Excel/VBA program to evaluate ex using an infi nite series. (b) Evaluation of ex. (c) Evaluation of e2x.

cha9792x_ch03_055-080.indd Page 78 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 78 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 79

sum are much larger than the fi nal result of the sum. Furthermore, unlike the previous case, the individual terms vary in sign. Thus, in effect we are adding and subtracting large num- bers (each with some small error) and placing great signifi cance on the differences—that is, subtractive cancellation. Thus, we can see that the culprit behind this example of smear- ing is, in fact, subtractive cancellation. For such cases it is appropriate to seek some other computational strategy. For example, one might try to compute y 5 e10 as y 5 (e21)10. Other than such a reformulation, the only general recourse is extended precision.

Inner Products. As should be clear from the last sections, some infi nite series are particularly prone to round-off error. Fortunately, the calculation of series is not one of the more common operations in numerical methods. A far more ubiquitous manipulation is the calculation of inner products, as in

a n

i51 xi yi 5 x1 y1 1 x2 y2 1 p 1 xn yn

This operation is very common, particularly in the solution of simultaneous linear alge- braic equations. Such summations are prone to round-off error. Consequently, it is often desirable to compute such summations in extended precision. Although the foregoing sections should provide rules of thumb to mitigate round-off error, they do not provide a direct means beyond trial and error to actually determine the effect of such errors on a computation. In Chap. 4, we will introduce the Taylor series, which will provide a mathematical approach for estimating these effects.

PROBLEMS

3.1 Convert the following base-2 numbers to base-10: (a) 101101, (b) 101.011, and (c) 0.01101. 3.2 Convert the following base-8 numbers to base-10: 71,263 and 3.147. 3.3 Compose your own program based on Fig. 3.11 and use it to determine your computer’s machine epsilon. 3.4 In a fashion similar to that in Fig. 3.11, write a short program to determine the smallest number, xmin, used on the computer you will be employing along with this book. Note that your computer will be unable to reliably distinguish between zero and a quantity that is smaller than this number. 3.5 The infi nite series

f (n) 5 a n

i51

1

i4

converges on a value of f (n) 5 p4y90 as n approaches infi nity. Write a program in single precision to calculate f (n) for n 5 10,000 by computing the sum from i 5 1 to 10,000. Then repeat the calcu- lation but in reverse order—that is, from i 5 10,000 to 1 using incre- ments of 21. In each case, compute the true percent relative error. Explain the results.

3.6 Evaluate e25 using two approaches

e2x 5 1 2 x 1 x2

2 2

x3

3! 1 p

and

e2x 5 1

ex 5

1

1 1 x 1 x2

2 1

x3

3! 1 p

and compare with the true value of 6.737947 3 1023. Use 20 terms to evaluate each series and compute true and approximate relative errors as terms are added. 3.7 The derivative of f (x) 5 1y(1 2 3x2) is given by

6x

(1 2 3x2)2

Do you expect to have difficulties evaluating this function at x 5 0.577? Try it using 3- and 4-digit arithmetic with chopping. 3.8 (a) Evaluate the polynomial

y 5 x3 2 5x2 1 6x 1 0.55

cha9792x_ch03_055-080.indd Page 79 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 79 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

80 APPROXIMATIONS AND ROUND-OFF ERRORS

at x 5 1.37. Use 3-digit arithmetic with chopping. Evaluate the percent relative error. (b) Repeat (a) but express y as

y 5 ( (x 2 5)x 1 6)x 1 0.55

Evaluate the error and compare with part (a). 3.9 Calculate the random access memory (RAM) in megabytes necessary to store a multidimensional array that is 20 3 40 3 120. This array is double precision, and each value requires a 64-bit word. Recall that a 64-bit word 5 8 bytes and 1 kilobyte 5 210 bytes. Assume that the index starts at 1. 3.10 Determine the number of terms necessary to approximate cos x to 8 signifi cant fi gures using the Maclaurin series approximation

cos x 5 1 2 x2

2 1

x4

4! 2

x6

6! 1

x8

8! 2 p

Calculate the approximation using a value of x 5 0.3p. Write a program to determine your result.

3.11 Use 5-digit arithmetic with chopping to determine the roots of the following equation with Eqs. (3.12) and (3.13)

x2 2 5000.002x 1 10

Compute percent relative errors for your results. 3.12 How can the machine epsilon be employed to formulate a stopping criterion es for your programs? Provide an example. 3.13 The “divide and average” method, an old-time method for approximating the square root of any positive number a, can be formulated as

x 5 x 1 ayx

2

Write a well-structured function to implement this algorithm based on the algorithm outlined in Fig. 3.3.

cha9792x_ch03_055-080.indd Page 80 09/10/13 7:32 PM F-468 cha9792x_ch03_055-080.indd Page 80 09/10/13 7:32 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4 C H A P T E R 4

81

Truncation Errors and the Taylor Series

Truncation errors are those that result from using an approximation in place of an exact mathematical procedure. For example, in Chap. 1 we approximated the deriva- tive of velocity of a falling parachutist by a fi nite-divided-difference equation of the form [Eq. (1.11)]

dy

dt > ¢y ¢t

5 y(ti11) 2 y(ti)

ti11 2 ti (4.1)

A truncation error was introduced into the numerical solution because the difference equation only approximates the true value of the derivative (recall Fig. 1.4). In order to gain insight into the properties of such errors, we now turn to a mathematical formulation that is used widely in numerical methods to express functions in an approximate fashion— the Taylor series.

4.1 THE TAYLOR SERIES

Taylor’s theorem (Box 4.1) and its associated formula, the Taylor series, is of great value in the study of numerical methods. In essence, the Taylor series provides a means to predict a function value at one point in terms of the function value and its deriva- tives at another point. In particular, the theorem states that any smooth function can be approximated as a polynomial. A useful way to gain insight into the Taylor series is to build it term by term. For example, the fi rst term in the series is

f(xi11) > f(xi) (4.2)

This relationship, called the zero-order approximation, indicates that the value of f at the new point is the same as its value at the old point. This result makes intuitive sense because if xi and xi+1 are close to each other, it is likely that the new value is probably similar to the old value. Equation (4.2) provides a perfect estimate if the function being approximated is, in fact, a constant. However, if the function changes at all over the interval, additional terms

cha9792x_ch04_081-109.indd Page 81 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 81 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

82 TRUNCATION ERRORS AND THE TAYLOR SERIES

Box 4.1 Taylor’s Theorem

Taylor’s Theorem If the function f and its fi rst n 1 1 derivatives are continuous on an in- terval containing a and x, then the value of the function at x is given by

f(x) 5 f(a) 1 f ¿(a) (x 2 a) 1 f –(a)

2! (x 2 a)2

1 f (3)(a)

3! (x 2 a)3 1 p

1 f (n)(a)

n! (x 2 a)n 1 Rn (B4.1.1)

where the remainder Rn is defi ned as

Rn 5 # x

a

(x 2 t)n

n! f (n11)(t)dt (B4.1.2)

where t 5 a dummy variable. Equation (B4.1.1) is called the Taylor series or Taylor’s formula. If the remainder is omitted, the right side of Eq. (B4.1.1) is the Taylor polynomial approximation to f(x). In essence, the theorem states that any smooth function can be ap- proximated as a polynomial. Equation (B4.1.2) is but one way, called the integral form, by which the remainder can be expressed. An alternative formulation can be derived on the basis of the integral mean-value theorem.

First Theorem of Mean for Integrals If the function g is continuous and integrable on an interval contain- ing a and x, then there exists a point j between a and x such that

# x

a

g(t) dt 5 g(j) (x 2 a) (B4.1.3)

In other words, this theorem states that the integral can be repre- sented by an average value for the function g(j) times the interval length x 2 a. Because the average must occur between the mini- mum and maximum values for the interval, there is a point x 5 j at which the function takes on the average value. The fi rst theorem is in fact a special case of a second mean- value theorem for integrals.

Second Theorem of Mean for Integrals If the functions g and h are continuous and integrable on an interval containing a and x, and h does not change sign in the interval, then there exists a point j between a and x such that

# x

a

g(t)h(t)dt 5 g(j) # x

a

h(t) dt (B4.1.4)

Thus, Eq. (B4.1.3) is equivalent to Eq. (B4.1.4) with h(t) 5 1. The second theorem can be applied to Eq. (B4.1.2) with

g(t) 5 f (n11)(t)  h(t) 5 (x 2 t)n

n!

As t varies from a to x, h(t) is continuous and does not change sign. Therefore, if f (n11)(t) is continuous, then the integral mean-value theorem holds and

Rn 5 f (n11)(j)

(n 1 1)! (x 2 a)n11

This equation is referred to as the derivative or Lagrange form of the remainder.

of the Taylor series are required to provide a better estimate. For example, the fi rst-order approximation is developed by adding another term to yield

f(xi11) > f(xi) 1 f ¿(xi)(xi11 2 xi) (4.3)

The additional fi rst-order term consists of a slope f 9(xi) multiplied by the distance between xi and xi+1. Thus, the expression is now in the form of a straight line and is capable of predicting an increase or decrease of the function between xi and xi+1. Although Eq. (4.3) can predict a change, it is exact only for a straight-line, or linear, trend. Therefore, a second-order term is added to the series to capture some of the cur- vature that the function might exhibit:

f(xi11) > f(xi) 1 f ¿(xi)(xi11 2 xi) 1 f –(xi)

2! (xi11 2 xi)

2 (4.4)

cha9792x_ch04_081-109.indd Page 82 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 82 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 83

In a similar manner, additional terms can be included to develop the complete Taylor series expansion:

f(xi11) 5 f(xi) 1 f ¿(xi)(xi11 2 xi) 1 f –(xi)

2! (xi11 2 xi)

2

1 f (3)(xi)

3! (xi11 2 xi)

3 1 p 1 f (n)(xi)

n! (xi11 2 xi)

n 1 Rn (4.5)

Note that because Eq. (4.5) is an infi nite series, an equal sign replaces the approximate sign that was used in Eqs. (4.2) through (4.4). A remainder term is included to account for all terms from n 1 1 to infi nity:

Rn 5 f (n11)(j)

(n 1 1)! (xi11 2 xi)

n11 (4.6)

where the subscript n connotes that this is the remainder for the nth-order approximation and j is a value of x that lies somewhere between xi and xi+1. The introduction of the j is so important that we will devote an entire section (Sec. 4.1.1) to its derivation. For the time being, it is suffi cient to recognize that there is such a value that provides an exact determination of the error. It is often convenient to simplify the Taylor series by defi ning a step size h 5 xi+1 2 xi and expressing Eq. (4.5) as

f(xi11) 5 f(xi) 1 f ¿(xi)h 1 f –(xi)

2! h2 1

f (3)(xi)

3! h3 1 p 1

f n(xi)

n! hn 1 Rn (4.7)

where the remainder term is now

Rn 5 f (n11)(j)

(n 1 1)! hn11 (4.8)

EXAMPLE 4.1 Taylor Series Approximation of a Polynomial

Problem Statement. Use zero- through fourth-order Taylor series expansions to approxi- mate the function

f (x) 5 20.1x4 2 0.15x3 2 0.5x2 2 0.25x 1 1.2

from xi 5 0 with h 5 1. That is, predict the function’s value at xi+1 5 1.

Solution. Because we are dealing with a known function, we can compute values for f(x) between 0 and 1. The results (Fig. 4.1) indicate that the function starts at f (0) 5 1.2 and then curves downward to f (1) 5 0.2. Thus, the true value that we are trying to predict is 0.2. The Taylor series approximation with n 5 0 is [Eq. (4.2)]

f(xi11) . 1.2

cha9792x_ch04_081-109.indd Page 83 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 83 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

84 TRUNCATION ERRORS AND THE TAYLOR SERIES

Thus, as in Fig. 4.1, the zero-order approximation is a constant. Using this formulation results in a truncation error [recall Eq. (3.2)] of

Et 5 0.2 2 1.2 5 21.0

at x 5 1. For n 5 1, the fi rst derivative must be determined and evaluated at x 5 0:

f ¿(0) 5 20.4(0.0)3 2 0.45(0.0)2 2 1.0(0.0) 2 0.25 5 20.25

Therefore, the fi rst-order approximation is [Eq. (4.3)]

f(xi11) . 1.2 2 0.25h

which can be used to compute f(1) 5 0.95. Consequently, the approximation begins to capture the downward trajectory of the function in the form of a sloping straight line (Fig. 4.1). This results in a reduction of the truncation error to

Et 5 0.2 2 0.95 5 20.75

For n 5 2, the second derivative is evaluated at x 5 0:

f –(0) 5 21.2(0.0)2 2 0.9(0.0) 2 1.0 5 21.0

Therefore, according to Eq. (4.4),

f(xi11) . 1.2 2 0.25h 2 0.5h 2

and substituting h 5 1, f (1) 5 0.45. The inclusion of the second derivative now adds some downward curvature resulting in an improved estimate, as seen in Fig. 4.1. The truncation error is reduced further to 0.2 2 0.45 5 20.25.

FIGURE 4.1 The approximation of f (x) 5 20.1x4 2 0.15x3 2 0.5x2 2 0.25x 1 1.2 at x 5 1 by zero- order, fi rst-order, and second-order Taylor series expansions.

Second order

First order

True

f (x)

1.0

0.5

0 xi = 0 xi + 1 = 1 x

f (xi + 1)

f (xi + 1) � f (xi) + f �(xi)h + h2

h

f �(xi) 2!

f (xi + 1) � f (xi) + f �(xi)h

f (xi + 1) � f (xi) f (xi) Zero order

cha9792x_ch04_081-109.indd Page 84 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 84 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 85

Additional terms would improve the approximation even more. In fact, the inclusion of the third and the fourth derivatives results in exactly the same equation we started with:

f(x) 5 1.2 2 0.25h 2 0.5h2 2 0.15h3 2 0.1h4

where the remainder term is

R4 5 f (5)(j)

5! h5 5 0

because the fi fth derivative of a fourth-order polynomial is zero. Consequently, the Taylor series expansion to the fourth derivative yields an exact estimate at xi+1 5 1:

f(1) 5 1.2 2 0.25(1) 2 0.5(1)2 2 0.15(1)3 2 0.1(1)4 5 0.2

In general, the nth-order Taylor series expansion will be exact for an nth-order polynomial. For other differentiable and continuous functions, such as exponentials and sinusoids, a fi nite number of terms will not yield an exact estimate. Each additional term will contribute some improvement, however slight, to the approximation. This behavior will be demonstrated in Example 4.2. Only if an infi nite number of terms are added will the series yield an exact result. Although the above is true, the practical value of Taylor series expansions is that, in most cases, the inclusion of only a few terms will result in an approximation that is close enough to the true value for practical purposes. The assessment of how many terms are required to get “close enough” is based on the remainder term of the expansion. Recall that the remainder term is of the general form of Eq. (4.8). This relationship has two major drawbacks. First, j is not known exactly but merely lies somewhere between xi and xi+1. Second, to evaluate Eq. (4.8), we need to determine the (n 1 1)th derivative of f(x). To do this, we need to know f(x). However, if we knew f(x), there would be no need to perform the Taylor series expansion in the present context! Despite this dilemma, Eq. (4.8) is still useful for gaining insight into truncation errors. This is because we do have control over the term h in the equation. In other words, we can choose how far away from x we want to evaluate f(x), and we can control the num- ber of terms we include in the expansion. Consequently, Eq. (4.8) is usually expressed as

Rn 5 O(hn11)

where the nomenclature O(hn11) means that the truncation error is of the order of hn11. That is, the error is proportional to the step size h raised to the (n 1 l)th power. Although this approximation implies nothing regarding the magnitude of the derivatives that multiply hn11, it is extremely useful in judging the comparative error of numerical methods based on Taylor series expansions. For example, if the error is O(h), halving the step size will halve the error. On the other hand, if the error is O(h2), halving the step size will quarter the error. In general, we can usually assume that the truncation error is decreased by the ad- dition of terms to the Taylor series. In many cases, if h is suffi ciently small, the fi rst- and other lower-order terms usually account for a disproportionately high percent of the error. Thus, only a few terms are required to obtain an adequate estimate. This property is illustrated by the following example.

cha9792x_ch04_081-109.indd Page 85 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 85 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

86 TRUNCATION ERRORS AND THE TAYLOR SERIES

EXAMPLE 4.2 Use of Taylor Series Expansion to Approximate a Function with an Infi nite Number of Derivatives

Problem Statement. Use Taylor series expansions with n 5 0 to 6 to approximate f(x) 5 cos x at xi+1 5 py3 on the basis of the value of f(x) and its derivatives at xi 5 py4. Note that this means that h 5 py3 2 py4 5 py12.

Solution. As with Example 4.1, our knowledge of the true function means that we can determine the correct value f(py3) 5 0.5. The zero-order approximation is [Eq. (4.3)]

f ap 3 b > cos ap

4 b 5 0.707106781

which represents a percent relative error of

et 5 0.5 2 0.707106781

0.5 100% 5 241.4%

For the fi rst-order approximation, we add the fi rst derivative term where f 9(x) 5 2sin x:

f ap 3 b > cos ap

4 b 2 sin ap

4 b a p

12 b 5 0.521986659

which has et 5 24.40 percent. For the second-order approximation, we add the second derivative term where f 0(x) 5 2cos x:

f ap 3 b > cos ap

4 b 2 sin ap

4 b a p

12 b 2 cos(py4)

2 a p

12 b2 5 0.497754491

with et 5 0.449 percent. Thus, the inclusion of additional terms results in an improved estimate. The process can be continued and the results listed, as in Table 4.1. Notice that the derivatives never go to zero, as was the case with the polynomial in Example 4.1. There- fore, each additional term results in some improvement in the estimate. However, also notice how most of the improvement comes with the initial terms. For this case, by the time we have added the third-order term, the error is reduced to 2.62 3 1022 percent,

TABLE 4.1 Taylor series approximation of f(x) 5 cos x at xi11 5 p/3 using a base point of p/4. Values are shown for various orders (n) of approximation.

Order n f (n)(x) f(P/3) Et

0 cos x 0.707106781 241.4 1 2sin x 0.521986659 24.4 2 2cos x 0.497754491 0.449 3 sin x 0.499869147 2.62 3 1022

4 cos x 0.500007551 21.51 3 1023

5 2sin x 0.500000304 26.08 3 1025

6 2cos x 0.499999988 2.44 3 1026

cha9792x_ch04_081-109.indd Page 86 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 86 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 87

4.1.1 The Remainder for the Taylor Series Expansion

Before demonstrating how the Taylor series is actually used to estimate numerical errors, we must explain why we included the argument j in Eq. (4.8). A mathematical derivation is presented in Box 4.1. We will now develop an alternative exposition based on a some- what more visual interpretation. Then we can extend this specifi c case to the more general formulation. Suppose that we truncated the Taylor series expansion [Eq. (4.7)] after the zero- order term to yield

f(xi11) > f(xi)

A visual depiction of this zero-order prediction is shown in Fig. 4.2. The remainder, or error, of this prediction, which is also shown in the illustration, consists of the infi nite series of terms that were truncated:

R0 5 f ¿(xi)h 1 f –(xi)

2! h2 1

f (3)(xi)

3! h3 1 p

It is obviously inconvenient to deal with the remainder in this infi nite series format. One simplifi cation might be to truncate the remainder itself, as in

R0 > f ¿(xi)h (4.9)

FIGURE 4.2 Graphical depiction of a zero-order Taylor series prediction and remainder.

Zero-order prediction

Exa ct p

red icti

on

f (x)

xi xi + 1 x

h

f (xi)

R0

which means that we have attained 99.9738 percent of the true value. Consequently, although the addition of more terms will reduce the error further, the improvement becomes negligible.

cha9792x_ch04_081-109.indd Page 87 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 87 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

88 TRUNCATION ERRORS AND THE TAYLOR SERIES

Although, as stated in the previous section, lower-order derivatives usually account for a greater share of the remainder than the higher-order terms, this result is still inexact because of the neglected second- and higher-order terms. This “inexactness” is implied by the approximate equality symbol (>) employed in Eq. (4.9). An alternative simplifi cation that transforms the approximation into an equivalence is based on a graphical insight. As in Fig. 4.3, the derivative mean-value theorem states that if a function f(x) and its fi rst derivative are continuous over an interval from xi to xi+1, then there exists at least one point on the function that has a slope, designated by f9(j), that is parallel to the line joining f(xi) and f(xi+1). The parameter j marks the x value where this slope occurs (Fig. 4.3). A physical illustration of this theorem is that, if you travel between two points with an average velocity, there will be at least one mo- ment during the course of the trip when you will be moving at that average velocity. By invoking this theorem it is simple to realize that, as illustrated in Fig. 4.3, the slope f9(j) is equal to the rise R0 divided by the run h, or

f ¿(j) 5 R0 h

which can be rearranged to give

R0 5 f ¿(j)h (4.10)

Thus, we have derived the zero-order version of Eq. (4.8). The higher-order versions are merely a logical extension of the reasoning used to derive Eq. (4.10). The fi rst-order version is

R1 5 f –(j)

2! h2 (4.11)

FIGURE 4.3 Graphical depiction of the derivative mean-value theorem.

f (x)

xi xi + 1� x

h

R0

Slope = f �(� )

Slope = R0 h

cha9792x_ch04_081-109.indd Page 88 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 88 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 89

For this case, the value of j conforms to the x value corresponding to the second de- rivative that makes Eq. (4.11) exact. Similar higher-order versions can be developed from Eq. (4.8).

4.1.2 Using the Taylor Series to Estimate Truncation Errors

Although the Taylor series will be extremely useful in estimating truncation errors throughout this book, it may not be clear to you how the expansion can actually be applied to numerical methods. In fact, we have already done so in our example of the falling parachutist. Recall that the objective of both Examples 1.1 and 1.2 was to pre- dict velocity as a function of time. That is, we were interested in determining y(t). As specifi ed by Eq. (4.5), y(t) can be expanded in a Taylor series:

y(ti11) 5 y(ti) 1 y¿(ti)(ti11 2 ti) 1 y–(ti)

2! (ti11 2 ti)

2 1 p 1 Rn (4.12)

Now let us truncate the series after the fi rst derivative term:

y(ti11) 5 y(ti) 1 y¿(ti)(ti11 2 ti) 1 R1 (4.13)

Equation (4.13) can be solved for

y¿(ti) 5 y(ti11) 2 y(ti)

ti11 2 ti 2

R1 ti11 2 ti

(4.14)

First-order Truncation approximation error

The fi rst part of Eq. (4.14) is exactly the same relationship that was used to approximate the derivative in Example 1.2 [Eq. (1.11)]. However, because of the Taylor series ap- proach, we have now obtained an estimate of the truncation error associated with this approximation of the derivative. Using Eqs. (4.6) and (4.14) yields

R1 ti11 2 ti

5 y–(j)

2! (ti11 2 ti) (4.15)

or

R1 ti11 2 ti

5 O(ti11 2 ti) (4.16)

Thus, the estimate of the derivative [Eq. (1.11) or the fi rst part of Eq. (4.14)] has a trun- cation error of order ti11 2 ti. In other words, the error of our derivative approximation should be proportional to the step size. Consequently, if we halve the step size, we would expect to halve the error of the derivative.

EXAMPLE 4.3 The Effect of Nonlinearity and Step Size on the Taylor Series Approximation

Problem Statement. Figure 4.4 is a plot of the function

f(x) 5 xm (E4.3.1)

for m 5 1, 2, 3, and 4 over the range from x 5 1 to 2. Notice that for m 5 1 the function is linear, and as m increases, more curvature or nonlinearity is introduced into the function.

cha9792x_ch04_081-109.indd Page 89 09/10/13 7:33 PM F-468 cha9792x_ch04_081-109.indd Page 89 09/10/13 7:33 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

90 TRUNCATION ERRORS AND THE TAYLOR SERIES

FIGURE 4.4 Plot of the function f (x) 5 xm for m 5 1, 2, 3, and 4. Notice that the function becomes more nonlinear as m increases.

1 0

5

10

15

2 x

f (x)

m = 2

m =

3

m =

4

m = 1

Employ the fi rst-order Taylor series to approximate this function for various values of the exponent m and the step size h.

Solution. Equation (E4.3.1) can be approximated by a fi rst-order Taylor series expansion, as in

f(xi11) 5 f(xi) 1 mx  m21 i h (E4.3.2)

which has a remainder

R1 5 f –(xi)

2! h2 1

f (3)(xi)

3! h3 1

f (4)(xi)

4! h4 1 p

First, we can examine how the approximation performs as m increases—that is, as the func- tion becomes more nonlinear. For m 5 1, the actual value of the function at x 5 2 is 2.

cha9792x_ch04_081-109.indd Page 90 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 90 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 91

The Taylor series yields

f(2) 5 1 1 1(1) 5 2

and

R1 5 0

The remainder is zero because the second and higher derivatives of a linear function are zero. Thus, as expected, the fi rst-order Taylor series expansion is perfect when the underlying function is linear. For m 5 2, the actual value is f (2) 5 22 5 4. The first-order Taylor series approximation is

f(2) 5 1 1 2(1) 5 3

and

R1 5 2 2(1)

2 1 0 1 0 1 p 5 1

Thus, because the function is a parabola, the straight-line approximation results in a discrepancy. Note that the remainder is determined exactly. For m 5 3, the actual value is f(2) 5 23 5 8. The Taylor series approximation is

f(2) 5 1 1 3(1)2(1) 5 4

and

R1 5 6 2(1)

2 1 66(1) 3 1 0 1 0 1 p 5 4

Again, there is a discrepancy that can be determined exactly from the Taylor series. For m 5 4, the actual value is f(2) 5 24 5 16. The Taylor series approximation is

f(2) 5 1 1 4(1)3(1) 5 5

and

R1 5 12 2 (1)

2 1 246 (1) 3 1 2424(1)

4 1 0 1 0 1 p 5 11

On the basis of these four cases, we observe that R1 increases as the function be- comes more nonlinear. Furthermore, R1 accounts exactly for the discrepancy. This is because Eq. (E4.3.1) is a simple monomial with a fi nite number of derivatives. This permits a complete determination of the Taylor series remainder. Next, we will examine Eq. (E4.3.2) for the case m 5 4 and observe how R1 changes as the step size h is varied. For m 5 4, Eq. (E4.3.2) is

f(x 1 h) 5 f(x) 1 4x 3i h

If x 5 1, f(1) 5 1 and this equation can be expressed as

f(1 1 h) 5 1 1 4h

with a remainder of

R1 5 6h 2 1 4h3 1 h4

cha9792x_ch04_081-109.indd Page 91 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 91 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

92 TRUNCATION ERRORS AND THE TAYLOR SERIES

This leads to the conclusion that the discrepancy will decrease as h is reduced. Also, at suffi ciently small values of h, the error should become proportional to h2. That is, as h is halved, the error will be quartered. This behavior is confi rmed by Table 4.2 and Fig. 4.5. Thus, we conclude that the error of the fi rst-order Taylor series approximation decreases as m approaches 1 and as h decreases. Intuitively, this means that the Taylor

FIGURE 4.5 Log-log plot of the remainder R1 of the fi rst-order Taylor series approximation of the function f (x) 5 x4 versus step size h. A line with a slope of 2 is also shown to indicate that as h decreases, the error becomes proportional to h2.

�Slope� = 2

0.11 0.001

0.01

0.1

1

10

0.01 h

R1

TABLE 4.2 Comparison of the exact value of the function f (x) 5 x4 with the fi rst-order Taylor series approximation. Both the function and the approximation are evaluated at x 1 h, where x 5 1.

First-Order h True Approximation R1

1 16 5 11 0.5 5.0625 3 2.0625 0.25 2.441406 2 0.441406 0.125 1.601807 1.5 0.101807 0.0625 1.274429 1.25 0.024429 0.03125 1.130982 1.125 0.005982 0.015625 1.063980 1.0625 0.001480

cha9792x_ch04_081-109.indd Page 92 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 92 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 93

series becomes more accurate when the function we are approximating becomes more like a straight line over the interval of interest. This can be accomplished either by reduc- ing the size of the interval or by “straightening” the function by reducing m. Obviously, the latter option is usually not available in the real world because the functions we analyze are typically dictated by the physical problem context. Consequently, we do not have control of their lack of linearity, and our only recourse is reducing the step size or includ- ing additional terms in the Taylor series expansion.

4.1.3 Numerical Differentiation

Equation (4.14) is given a formal label in numerical methods—it is called a fi nite divided difference. It can be represented generally as

f ¿(xi) 5 f(xi11) 2 f(xi)

xi11 2 xi 1 O(xi11 2 xi) (4.17)

or

f ¿(xi) 5 ¢fi h

1 O(h) (4.18)

where D fi is referred to as the fi rst forward difference and h is called the step size, that is, the length of the interval over which the approximation is made. It is termed a “forward” difference because it utilizes data at i and i 1 1 to estimate the derivative (Fig. 4.6a). The entire term D fyh is referred to as a fi rst fi nite divided difference. This forward divided difference is but one of many that can be developed from the Taylor series to approximate derivatives numerically. For example, backward and centered difference approximations of the fi rst derivative can be developed in a fashion similar to the derivation of Eq. (4.14). The former utilizes values at xi21 and xi (Fig. 4.6b), whereas the latter uses values that are equally spaced around the point at which the derivative is estimated (Fig. 4.6c). More accurate approximations of the fi rst derivative can be devel- oped by including higher-order terms of the Taylor series. Finally, all the above versions can also be developed for second, third, and higher derivatives. The following sections provide brief summaries illustrating how some of these cases are derived.

Backward Difference Approximation of the First Derivative. The Taylor series can be expanded backward to calculate a previous value on the basis of a present value, as in

f(xi21) 5 f(xi) 2 f ¿(xi)h 1 f –(xi)

2! h2 2 p (4.19)

Truncating this equation after the fi rst derivative and rearranging yields

f ¿(xi) > f(xi) 2 f(xi21)

h 5 § fi h

(4.20)

where the error is O(h), and = fi is referred to as the fi rst backward difference. See Fig. 4.6b for a graphical representation.

cha9792x_ch04_081-109.indd Page 93 09/12/13 9:09 AM F-468 cha9792x_ch04_081-109.indd Page 93 09/12/13 9:09 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

94 TRUNCATION ERRORS AND THE TAYLOR SERIES

FIGURE 4.6 Graphical depiction of (a) forward, (b) backward, and (c) centered fi nite-divided-difference approximations of the fi rst derivative.

2h

xi–1 xi+1 x

f (x)

Tru e d

eriv ativ

e

Ap pro

xim atio

n

(c)

h

xi–1 xi x

f (x)

Tru e d

eriv ativ

e

Ap pr

ox im

at io

n

(b)

h

xi xi+1 x

f (x)

Tru e d

eriv ativ

e

Approx imation

(a)

cha9792x_ch04_081-109.indd Page 94 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 94 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.1 THE TAYLOR SERIES 95

Centered Difference Approximation of the First Derivative. A third way to approxi- mate the fi rst derivative is to subtract Eq. (4.19) from the forward Taylor series expansion:

f  (xi11) 5 f  (xi) 1 f ¿(xi)h 1 f –(xi)

2! h2 1 p (4.21)

to yield

f  (xi11) 5 f  (xi21) 1 2f ¿(xi)h 1 2f (3)(xi)

3! h3 1 p

which can be solved for

f ¿(xi) 5 f  (xi11) 2 f  (xi21)

2h 2

f (3)(xi)

6 h2 2 p

or

f ¿(xi) 5 f  (xi11) 2 f  (xi21)

2h 2 O(h2) (4.22)

Equation (4.22) is a centered difference representation of the fi rst derivative. Notice that the truncation error is of the order of h2 in contrast to the forward and backward approximations that were of the order of h. Consequently, the Taylor series analysis yields the practical information that the centered difference is a more accurate represen- tation of the derivative (Fig. 4.6c). For example, if we halve the step size using a forward or backward difference, we would approximately halve the truncation error, whereas for the central difference, the error would be quartered.

EXAMPLE 4.4 Finite-Divided-Difference Approximations of Derivatives

Problem Statement. Use forward and backward difference approximations of O(h) and a centered difference approximation of O(h2) to estimate the fi rst derivative of

f  (x) 5 20.1x4 2 0.15x3 2 0.5x2 2 0.25x 1 1.25

at x 5 0.5 using a step size h 5 0.5. Repeat the computation using h 5 0.25. Note that the derivative can be calculated directly as

f ¿(x) 5 20.4x3 2 0.45x2 2 1.0x 2 0.25

and can be used to compute the true value as f9(0.5) 5 20.9125.

Solution. For h 5 0.5, the function can be employed to determine

xi21 5 0  f  (xi21) 5 1.2

xi 5 0.5  f  (xi) 5 0.925

xi11 5 1.0  f  (xi11) 5 0.2

These values can be used to compute the forward divided difference [Eq. (4.17)],

f  ¿(0.5) > 0.2 2 0.925

0.5 5 21.45  Zet Z 5 58.9%

cha9792x_ch04_081-109.indd Page 95 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 95 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

96 TRUNCATION ERRORS AND THE TAYLOR SERIES

the backward divided difference [Eq. (4.20)],

f ¿(0.5) > 0.925 2 1.2

0.5 5 20.55  Zet Z 5 39.7%

and the centered divided difference [Eq. (4.22)],

f ¿(0.5) > 0.2 2 1.2

1.0 5 21.0  Zet Z 5 9.6%

For h 5 0.25,

xi21 5 0.25  f  (xi21) 5 1.10351563

xi 5 0.5  f  (xi) 5 0.925

xi11 5 0.75  f  (xi11) 5 0.63632813

which can be used to compute the forward divided difference,

f ¿(0.5) > 0.63632813 2 0.925

0.25 5 21.155  Zet Z 5 26.5%

the backward divided difference,

f ¿(0.5) > 0.925 2 1.10351563

0.25 5 20.714  Zet Z 5 21.7%

and the centered divided difference,

f ¿(0.5) > 0.63632813 2 1.10351563

0.5 5 20.934  Zet Z 5 2.4%

For both step sizes, the centered difference approximation is more accurate than forward or backward differences. Also, as predicted by the Taylor series analysis, halving the step size approximately halves the error of the backward and forward differences and quarters the error of the centered difference.

Finite Difference Approximations of Higher Derivatives. Besides fi rst derivatives, the Taylor series expansion can be used to derive numerical estimates of higher deriva- tives. To do this, we write a forward Taylor series expansion for f(xi12) in terms of f(xi):

f (xi12) 5 f (xi) 1 f ¿(xi)(2h) 1 f –(xi)

2! (2h)2 1 p (4.23)

Equation (4.21) can be multiplied by 2 and subtracted from Eq. (4.23) to give

f  (xi12) 2 2f  (xi11) 5 2f  (xi) 1 f –(xi)h2 1 p

which can be solved for

f –(xi) 5 f  (xi12) 2 2f  (xi11) 1 f  (xi)

h2 1 O(h) (4.24)

cha9792x_ch04_081-109.indd Page 96 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 96 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.2 ERROR PROPAGATION 97

This relationship is called the second forward fi nite divided difference. Similar manipula- tions can be employed to derive a backward version

f –(xi) 5 f  (xi) 2 2f  (xi21) 1 f  (xi22)

h2 1 O(h)

and a centered version

f –(xi) 5 f  (xi11) 2 2f  (xi) 1 f  (xi21)

h2 1 O(h2)

As was the case with the fi rst-derivative approximations, the centered case is more accurate. Notice also that the centered version can be alternatively expressed as

f –(xi) >

f  (xi11) 2 f  (xi)

h 2

f  (xi) 2 f  (xi21)

h

h

Thus, just as the second derivative is a derivative of a derivative, the second divided difference approximation is a difference of two fi rst divided differences. We will return to the topic of numerical differentiation in Chap. 23. We have intro- duced you to the topic at this point because it is a very good example of why the Taylor series is important in numerical methods. In addition, several of the formulas introduced in this section will be employed prior to Chap. 23.

4.2 ERROR PROPAGATION

The purpose of this section is to study how errors in numbers can propagate through mathematical functions. For example, if we multiply two numbers that have errors, we would like to estimate the error in the product.

4.2.1 Functions of a Single Variable

Suppose that we have a function f(x) that is dependent on a single independent variable x. Assume that x̃ is an approximation of x. We, therefore, would like to assess the effect of the discrepancy between x and x̃ on the value of the function. That is, we would like to estimate

¢ f  (x̃) 5 Z f (x) 2 f (x̃)Z

The problem with evaluating ¢ f(x̃) is that f(x) is unknown because x is unknown. We can overcome this diffi culty if x̃ is close to x and f(x̃) is continuous and differentiable. If these conditions hold, a Taylor series can be employed to compute f(x) near f(x̃), as in

f(x) 5 f(x̃) 1 f ¿(x̃) (x 2 x̃) 1 f –(x̃)

2 (x 2 x̃)2 1 p

Dropping the second- and higher-order terms and rearranging yields

f  (x) 2 f  (x̃) > f ¿(x̃) (x 2 x̃)

cha9792x_ch04_081-109.indd Page 97 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 97 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

98 TRUNCATION ERRORS AND THE TAYLOR SERIES

or

¢ f (x̃) 5 Z f ¿(x̃) Z¢x̃ (4.25)

where ¢ f  (x̃) 5 Z f  (x) 2 f  (x̃) Z represents an estimate of the error of the function and ¢x̃ 5 Zx 2 x̃ Z represents an estimate of the error of x. Equation (4.25) provides the capabil- ity to approximate the error in f(x) given the derivative of a function and an estimate of the error in the independent variable. Figure 4.7 is a graphical illustration of the operation.

EXAMPLE 4.5 Error Propagation in a Function of a Single Variable

Problem Statement. Given a value of x̃ 5 2.5 with an error of ¢x̃ 5 0.01, estimate the resulting error in the function f(x) 5 x3.

Solution. Using Eq. (4.25),

¢ f(x̃) > 3(2.5)2(0.01) 5 0.1875

Because f(2.5) 5 15.625, we predict that

f  (2.5) 5 15.625 6 0.1875

or that the true value lies between 15.4375 and 15.8125. In fact, if x were actually 2.49, the function could be evaluated as 15.4382, and if x were 2.51, it would be 15.8132. For this case, the fi rst-order error analysis provides a fairly close estimate of the true error.

True error � f �(x)��x

Estimated error

x x x

f (x)

�x FIGURE 4.7 Graphical depiction of fi rst- order error propagation.

cha9792x_ch04_081-109.indd Page 98 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 98 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.2 ERROR PROPAGATION 99

4.2.2 Functions of More than One Variable

The foregoing approach can be generalized to functions that are dependent on more than one independent variable. This is accomplished with a multivariable version of the Taylor series. For example, if we have a function of two independent variables u and y, the Taylor series can be written as

f(ui11, yi11) 5 f(ui, yi) 1 0f 0u

(ui11 2 ui) 1 0f 0y

(yi11 2 yi)

1 1

2! c 02 f 0u2

(ui11 2 ui) 2 1 2

02 f 0u0y

(ui11 2 ui)(yi11 2 yi)

1 02 f 0y2

(yi11 2 yi) 2 d 1 p (4.26)

where all partial derivatives are evaluated at the base point i. If all second-order and higher terms are dropped, Eq. (4.26) can be solved for

¢ f  (ũ, ỹ) 5 ` 0f 0u ` ¢ũ 1 ` 0f

0y ` ¢ỹ

where ¢ũ and ¢ỹ 5 estimates of the errors in u and y, respectively. For n independent variables x̃1, x̃2, p , x̃n having errors ¢x̃1, ¢x̃2, p , ¢xn the following general relationship holds:

¢f  (x̃1, x̃2, p , x̃n) > ` 0f0x1 ` ¢x̃1 1 ` 0f 0x2 ` ¢x̃2 1 p 1 ` 0f0xn ` ¢x̃n (4.27)

EXAMPLE 4.6 Error Propagation in a Multivariable Function

Problem Statement. The defl ection y of the top of a sailboat mast is

y 5 FL4

8EI

where F 5 a uniform side loading (N/m), L 5 height (m), E 5 the modulus of elasticity (N/m2), and I 5 the moment of inertia (m4). Estimate the error in y given the following data:

F̃ 5 750 N/m  ¢F̃ 5 30 N/m L̃ 5 9 m  ¢L̃ 5 0.03 m Ẽ 5 7.5 3 109 N/m2  ¢Ẽ 5 5 3 107 N/m2

Ĩ 5 0.0005 m4   ¢Ĩ 5 0.000005 m4

Solution. Employing Eq. (4.27) gives

¢y(F̃, L̃, Ẽ, Ĩ ) 5 ` 0y 0F ` ¢F̃ 1 ` 0y

0L ` ¢L̃ 1 ` 0y

0E ` ¢Ẽ 1 ` 0y

0I ` ¢Ĩ

or

¢y(F̃, L̃, Ẽ, Ĩ 2 > L̃ 4

8Ẽ Ĩ ¢F̃ 1

F̃L̃3

2ẼĨ ¢L̃ 1

F̃L̃4

8Ẽ2 Ĩ ¢Ẽ 1

F̃L̃4

8Ẽ Ĩ 2 ¢ Ĩ

cha9792x_ch04_081-109.indd Page 99 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 99 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

100 TRUNCATION ERRORS AND THE TAYLOR SERIES

Substituting the appropriate values gives

¢y 5 0.006561 1 0.002187 1 0.001094 1 0.00164 5 0.011482

Therefore, y 5 0.164025 6 0.011482. In other words, y is between 0.152543 and 0.175507 m. The validity of these estimates can be verifi ed by substituting the extreme values for the variables into the equation to generate an exact minimum of

ymin 5 720(8.97)4

8(7.55 3 109)0.000505 5 0.152818

and

ymax 5 780(9.03)4

8(7.45 3 109)0.000495 5 0.175790

Thus, the fi rst-order estimates are reasonably close to the exact values.

Equation (4.27) can be employed to defi ne error propagation relationships for common mathematical operations. The results are summarized in Table 4.3. We will leave the derivation of these formulas as a homework exercise.

4.2.3 Stability and Condition

The condition of a mathematical problem relates to its sensitivity to changes in its input values. We say that a computation is numerically unstable if the uncertainty of the input values is grossly magnifi ed by the numerical method. These ideas can be studied using a fi rst-order Taylor series

f  (x) 5 f  (x̃) 1 f ¿(x̃) (x 2 x̃)

This relationship can be employed to estimate the relative error of f(x) as in

f  (x) 2 f  (x̃)

f  (x) >

f ¿(x̃) (x 2 x̃) f  (x̃)

The relative error of x is given by

x 2 x̃

TABLE 4.3 Estimated error bounds associated with common mathematical operations using inexact numbers ũ and ṽ.

Operation Estimated Error

Addition ¢ (ũ 1 ṽ) ¢ũ 1 ¢ṽ Subtraction ¢ (ũ 2 ṽ) ¢ũ 1 ¢ṽ Multiplication ¢ (ũ 3 ṽ) Z ũ Z¢ṽ 1 Z ṽ Z¢ũ

Division ¢ a ũ ṽ b Z ũ Z¢ṽ 1 Z ṽ Z¢ũ

Zṽ Z2

cha9792x_ch04_081-109.indd Page 100 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 100 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.3 TOTAL NUMERICAL ERROR 101

A condition number can be defi ned as the ratio of these relative errors

Condition number 5 x ˜ f ¿(x̃) f  (x̃)

(4.28)

The condition number provides a measure of the extent to which an uncertainty in x is magnifi ed by f(x). A value of 1 tells us that the function’s relative error is identical to the relative error in x. A value greater than 1 tells us that the relative error is amplifi ed, whereas a value less than 1 tells us that it is attenuated. Functions with very large values are said to be ill-conditioned. Any combination of factors in Eq. (4.28) that increases the numerical value of the condition number will tend to magnify uncertainties in the computation of f(x).

EXAMPLE 4.7 Condition Number

Problem Statement. Compute and interpret the condition number for

f(x) 5 tan x  for x̃ 5 p

2 1 0.1 ap

2 b

f(x) 5 tan x  for x̃ 5 p

2 1 0.01 ap

2 b

Solution. The condition number is computed as

Condition number 5 x̃(1ycos2 x)

tan x̃

For x̃ 5 py2 1 0.1(py2),

Condition number 5 1.7279(40.86)

26.314 5 211.2

Thus, the function is ill-conditioned. For x̃ 5 py2 1 0.01(py2), the situation is even worse:

Condition number 5 1.5865(4053)

263.66 5 2101

For this case, the major cause of ill conditioning appears to be the derivative. This makes sense because in the vicinity of py2, the tangent approaches both positive and negative infi nity.

4.3 TOTAL NUMERICAL ERROR

The total numerical error is the summation of the truncation and round-off errors. In general, the only way to minimize round-off errors is to increase the number of signifi cant fi gures of the computer. Further, we have noted that round-off error will increase due to subtractive cancellation or due to an increase in the number of computations in an analy- sis. In contrast, Example 4.4 demonstrated that the truncation error can be reduced by decreasing the step size. Because a decrease in step size can lead to subtractive cancella- tion or to an increase in computations, the truncation errors are decreased as the round-off

cha9792x_ch04_081-109.indd Page 101 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 101 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

102 TRUNCATION ERRORS AND THE TAYLOR SERIES

errors are increased. Therefore, we are faced by the following dilemma: The strategy for decreasing one component of the total error leads to an increase of the other component. In a computation, we could conceivably decrease the step size to minimize truncation errors only to discover that in doing so, the round-off error begins to dominate the solu- tion and the total error grows! Thus, our remedy becomes our problem (Fig. 4.8). One challenge that we face is to determine an appropriate step size for a particular computation. We would like to choose a large step size in order to decrease the amount of calculations and round-off errors without incurring the penalty of a large truncation error. If the total error is as shown in Fig. 4.8, the challenge is to identify the point of diminishing returns where round-off error begins to negate the benefi ts of step-size reduction. In actual cases, however, such situations are relatively uncommon because most com- puters carry enough signifi cant fi gures that round-off errors do not predominate. Neverthe- less, they sometimes do occur and suggest a sort of “numerical uncertainty principle” that places an absolute limit on the accuracy that may be obtained using certain computerized numerical methods. We explore such a case in the following section.

4.3.1 Error Analysis of Numerical Differentiation

As described in the Sec. 4.1.3, a centered difference approximation of the fi rst derivative can be written as (Eq. 4.22):

f ¿(xi) 5 f(xi11) 2 f(xi21)

2h 2

f (3)(j)

6 h2 (4.29)

True Finite-difference Truncation value approximation error

FIGURE 4.8 A graphical depiction of the trade-off between round-off and truncation error that sometimes comes into play in the course of a numerical method. The point of diminishing returns is shown, where round-off error begins to negate the benefi ts of step-size reduction.

Total error

Round-off error Tru

nca tio

n e rro

r

log step size

lo g

e rr

o r

Point of diminishing returns

cha9792x_ch04_081-109.indd Page 102 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 102 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.3 TOTAL NUMERICAL ERROR 103

Thus, if the two function values in the numerator of the fi nite-difference approximation have no round-off error, the only error is due to truncation. However, because we are using digital computers, the function values do include round-off error as in

f(xi21) 5 f̃(xi21) 1 ei21

f(xi11) 5 f̃(xi11) 1 ei11

where the f̃ ’s are the rounded function values and the e’s are the associated round-off errors. Substituting these values into Eq. (4.29) gives

f ¿(xi) 5 f̃ (xi11) 2 f̃ (xi21)

2h 1

ei11 2 ei21 2h

2 f (3)(j)

6 h2

True Finite-difference Round-off Truncation value approximation error error

We can see that the total error of the fi nite-difference approximation consists of a round- off error which increases with step size and a truncation error that decreases with step size. Assuming that the absolute value of each component of the round-off error has an upper bound of e, the maximum possible value of the difference ei+1 2 ei will be 2e. Further, assume that the third derivative has a maximum absolute value of M. An upper bound on the absolute value of the total error can therefore be represented as

Total error 5 ` f ¿(xi) 2 f̃  (xi11) 2 f̃ (xi21)2h ` # e

h 1

h2M

6 (4.30)

An optimal step size can be determined by differentiating Eq. (4.30), setting the result equal to zero and solving for

hopt 5 B 3 3e

M (4.31)

EXAMPLE 4.8 Round-off and Truncation Errors in Numerical Differentiation

Problem Statement. In Example 4.4, we used a centered difference approximation of O(h2) to estimate the fi rst derivative of the following function at x 5 0.5,

f(x) 5 20.1x4 2 0.15x3 2 0.5x2 2 0.25x 1 1.2

Perform the same computation starting with h 5 1. Then progressively divide the step size by a factor of 10 to demonstrate how round-off becomes dominant as the step size is reduced. Relate your results to Eq. (4.31). Recall that the true value of the derivative is 20.9125.

Solution. We can develop a program to perform the computations and plot the results. For the present example, we have done this with a MATLAB software M-fi le. Notice that we pass both the function and its analytical derivative as arguments. In addition, the function generates a plot of the results.

cha9792x_ch04_081-109.indd Page 103 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 103 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

104 TRUNCATION ERRORS AND THE TAYLOR SERIES

function diffex(func,dfunc,x,n) format long dftrue=dfunc(x); h=1; H(1)=h; D(1)=(func(x+h)−func(x−h))/(2*h); E(1)=abs(dftrue−D(1)); for i=2:n h=h/10; H(i)=h; D(i)=(func(x+h)−func(x−h))/(2*h); E(i)=abs(dftrue−D(i)); end L=[H' D' E']'; fprintf(' step size finite difference true error\n'); fprintf('%14.10f %16.14f %16.13f\n',L); loglog(H,E),xlabel('Step Size'),ylabel('Error') title('Plot of Error Versus Step Size') format short

The M-fi le can then be run using the following commands:

>> ff=@(x) −0.1*x^4−0.15*x^3−0.5*x^2−0.25*x+1.2; >> df=@(x) −0.4*x^3−0.45*x^2−x−0.25; >> diffex(ff,df,0.5,11)

When the function is run, the following numeric output is generated along with the plot (Fig. 4.9):

step size finite difference true error 1.0000000000 −1.26250000000000 0.3500000000000 0.1000000000 −0.91600000000000 0.0035000000000 0.0100000000 −0.91253500000000 0.0000350000000 0.0010000000 −0.91250035000001 0.0000003500000 0.0001000000 −0.91250000349985 0.0000000034998 0.0000100000 −0.91250000003318 0.0000000000332 0.0000010000 −0.91250000000542 0.0000000000054 0.0000001000 −0.91249999945031 0.0000000005497 0.0000000100 −0.91250000333609 0.0000000033361 0.0000000010 −0.91250001998944 0.0000000199894 0.0000000001 −0.91250007550059 0.0000000755006

The results are as expected. At fi rst, round-off is minimal and the estimate is dominated by truncation error. Hence, as in Eq. (4.30), the total error drops by a factor of 100 each time we divide the step by 10. However, starting at h 5 0.0001, we see round-off error begin to creep in and erode the rate at which the error diminishes. A minimum error is reached at h 5 1026. Beyond this point, the error increases as round-off dominates. Because we are dealing with an easily differentiable function, we can also investigate whether these results are consistent with Eq. (4.31). First, we can estimate M by evalu- ating the function’s third derivative as

M 5 Z f 3(0.5) Z 5 Z 22.4(0.5) 2 0.9 Z 5 2.1

cha9792x_ch04_081-109.indd Page 104 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 104 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.3 TOTAL NUMERICAL ERROR 105

Because MATLAB has a precision of about 15 to 16 base-10 digits, a rough estimate of the upper bound on round-off would be about e 5 0.5 3 10216. Substituting these values into Eq. (4.31) gives

hopt 5 B 3 3(0.5 3 10

216) 2.1

5 4.3 3 1026

which is on the same order as the result of 1 3 1026 obtained with our computer program.

4.3.2 Control of Numerical Errors

For most practical cases, we do not know the exact error associated with numerical meth- ods. The exception, of course, is when we have obtained the exact solution that makes our numerical approximations unnecessary. Therefore, for most engineering applications we must settle for some estimate of the error in our calculations. There are no systematic and general approaches to evaluating numerical errors for all problems. In many cases, error estimates are based on the experience and judgment of the engineer. Although error analysis is to a certain extent an art, there are several practical program- ming guidelines we can suggest. First and foremost, avoid subtracting two nearly equal numbers. Loss of signifi cance almost always occurs when this is done. Sometimes you can rearrange or reformulate the problem to avoid subtractive cancellation. If this is not pos- sible, you may want to use extended-precision arithmetic. Furthermore, when adding and

FIGURE 4.9 Plot of error versus step size.

E rr

o r

10–12 10–10 10–8 10–6 10–4

Step size

Plot of error versus step size

10–2 10–0

10–10

10–8

10–6

10–4

10–2

100

cha9792x_ch04_081-109.indd Page 105 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 105 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

106 TRUNCATION ERRORS AND THE TAYLOR SERIES

subtracting numbers, it is best to sort the numbers and work with the smallest numbers fi rst. This avoids loss of signifi cance. Beyond these computational hints, one can attempt to predict total numerical errors using theoretical formulations. The Taylor series is our primary tool for analysis of both truncation and round-off errors. Several examples have been presented in this chapter. Prediction of total numerical error is very complicated for even moderately sized problems and tends to be pessimistic. Therefore, it is usually attempted for only small-scale tasks. The tendency is to push forward with the numerical computations and try to estimate the accuracy of your results. This can sometimes be done by seeing if the results satisfy some condition or equation as a check. Or it may be possible to substitute the results back into the original equation to check that it is actually satisfi ed. Finally you should be prepared to perform numerical experiments to increase your awareness of computational errors and possible ill-conditioned problems. Such experi- ments may involve repeating the computations with a different step size or method and comparing the results. We may employ sensitivity analysis to see how our solution changes when we change model parameters or input values. We may want to try different nu- merical algorithms that have different theoretical foundations, are based on different com- putational strategies, or have different convergence properties and stability characteristics. When the results of numerical computations are extremely critical and may involve loss of human life or have severe economic ramifi cations, it is appropriate to take special precautions. This may involve the use of two or more independent groups to solve the same problem so that their results can be compared. The roles of errors will be a topic of concern and analysis in all sections of this book. We will leave these investigations to specifi c sections.

4.4 BLUNDERS, FORMULATION ERRORS, AND DATA UNCERTAINTY

Although the following sources of error are not directly connected with most of the numerical methods in this book, they can sometimes have great impact on the success of a modeling effort. Thus, they must always be kept in mind when applying numerical techniques in the context of real-world problems.

4.4.1 Blunders

We are all familiar with gross errors, or blunders. In the early years of computers, er- roneous numerical results could sometimes be attributed to malfunctions of the computer itself. Today, this source of error is highly unlikely, and most blunders must be attributed to human imperfection. Blunders can occur at any stage of the mathematical modeling process and can contribute to all the other components of error. They can be avoided only by sound knowledge of fundamental principles and by the care with which you approach and design your solution to a problem. Blunders are usually disregarded in discussions of numerical methods. This is no doubt due to the fact that, try as we may, mistakes are to a certain extent unavoidable. However, we believe that there are a number of ways in which their occurrence can be

cha9792x_ch04_081-109.indd Page 106 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 106 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

4.4 BLUNDERS, FORMULATION ERRORS, AND DATA UNCERTAINTY 107

minimized. In particular, the good programming habits that were outlined in Chap. 2 are extremely useful for mitigating programming blunders. In addition, there are usually simple ways to check whether a particular numerical method is working properly. Throughout this book, we discuss ways to check the results of numerical calculations.

4.4.2 Formulation Errors

Formulation, or model, errors relate to bias that can be ascribed to incomplete mathe- matical models. An example of a negligible formulation error is the fact that Newton’s second law does not account for relativistic effects. This does not detract from the ad- equacy of the solution in Example 1.1 because these errors are minimal on the time and space scales associated with the falling parachutist problem. However, suppose that air resistance is not linearly proportional to fall velocity, as in Eq. (1.7), but is a function of the square of velocity. If this were the case, both the analytical and numerical solutions obtained in the Chap. 1 would be erroneous because of formulation error. Further consideration of formulation error is included in some of the engineering applications in the remainder of the book. You should be cognizant of these problems and realize that, if you are working with a poorly conceived model, no numerical method will provide adequate results.

4.4.3 Data Uncertainty

Errors sometimes enter into an analysis because of uncertainty in the physical data upon which a model is based. For instance, suppose we wanted to test the falling parachutist model by having an individual make repeated jumps and then measuring his or her velocity after a specifi ed time interval. Uncertainty would undoubtedly be associated with these measurements, since the parachutist would fall faster during some jumps than during others. These errors can exhibit both inaccuracy and imprecision. If our instru- ments consistently underestimate or overestimate the velocity, we are dealing with an inaccurate, or biased, device. On the other hand, if the measurements are randomly high and low, we are dealing with a question of precision. Measurement errors can be quantifi ed by summarizing the data with one or more well-chosen statistics that convey as much information as possible regarding specifi c characteristics of the data. These descriptive statistics are most often selected to represent (1) the location of the center of the distribution of the data and (2) the degree of spread of the data. As such, they provide a measure of the bias and imprecision, respectively. We will return to the topic of characterizing data uncertainty in Part Five. Although you must be cognizant of blunders, formulation errors, and uncertain data, the numerical methods used for building models can be studied, for the most part, inde- pendently of these errors. Therefore, for most of this book, we will assume that we have not made gross errors, we have a sound model, and we are dealing with error-free mea- surements. Under these conditions, we can study numerical errors without complicating factors.

cha9792x_ch04_081-109.indd Page 107 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 107 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

108 TRUNCATION ERRORS AND THE TAYLOR SERIES

PROBLEMS

4.1 The following infi nite series can be used to approximate ex:

e x 5 1 1 x 1 x 2

2 1

x 3

3! 1 p 1

x n

n!

(a) Prove that this Maclaurin series expansion is a special case of the Taylor series expansion [(Eq. (4.7)] with xi 5 0 and h 5 x.

(b) Use the Taylor series to estimate f(x) 5 e2x at xi11 5 1 for xi 5 0.2. Employ the zero-, first-, second-, and third-order versions and compute the ZetZ for each case.

4.2 The Maclaurin series expansion for cos x is

cos x 5 1 2 x 2

2 1

x4

4! 2

x6

6! 1

x8

8! 2 p

Starting with the simplest version, cos x 5 1, add terms one at a time to estimate cos(py3). After each new term is added, compute the true and approximate percent relative errors. Use your pocket calculator to determine the true value. Add terms until the absolute value of the approximate error estimate falls below an error crite- rion conforming to two signifi cant fi gures. 4.3 Perform the same computation as in Prob. 4.2, but use the Maclaurin series expansion for the sin x to estimate sin(py3).

sin x 5 x 2 x3

3! 1

x5

5! 2

x7

7! 1 p

4.4 The Maclaurin series expansion for the arctangent of x is de- fi ned for Zx Z # 1 as

arctan x 5 a q

n50

(21)n

2n 1 1 x 2n11

(a) Write out the fi rst four terms (n 5 0, . . . , 3). (b) Starting with the simplest version, arctan x 5 x, add terms one

at a time to estimate arctan(py6). After each new term is added, compute the true and approximate percent relative errors. Use your calculator to determine the true value. Add terms until the absolute value of the approximate error estimate falls below an error criterion conforming to two signifi cant fi gures.

4.5 Use zero- through third-order Taylor series expansions to predict f (3) for

f (x) 5 25x3 2 6x2 1 7x 2 88

using a base point at x 5 1. Compute the true percent relative error et for each approximation. 4.6 Use zero- through fourth-order Taylor series expansions to pre- dict f(2.5) for f(x) 5 ln x using a base point at x 5 1. Compute the true percent relative error et for each approximation. Discuss the meaning of the results.

4.7 Use forward and backward difference approximations of O(h) and a centered difference approximation of O(h2) to estimate the fi rst derivative of the function examined in Prob. 4.5. Evaluate the derivative at x 5 2 using a step size of h 5 0.2. Compare your results with the true value of the derivative. Interpret your results on the basis of the remainder term of the Taylor series expansion. 4.8 Use a centered difference approximation of O(h2) to estimate the second derivative of the function examined in Prob. 4.5. Per- form the evaluation at x 5 2 using step sizes of h 5 0.25 and 0.125. Compare your estimates with the true value of the second deriva- tive. Interpret your results on the basis of the remainder term of the Taylor series expansion. 4.9 The Stefan-Boltzmann law can be employed to estimate the rate of radiation of energy H from a surface, as in

H 5 AesT  4

where H is in watts, A 5 the surface area (m2), e 5 the emissivity that characterizes the emitting properties of the surface (dimension- less), s 5 a universal constant called the Stefan-Boltzmann con- stant (5 5.67 3 1028 W m22 K24), and T 5 absolute temperature (K). Determine the error of H for a steel plate with A 5 0.15 m2, e 5 0.90, and T 5 650 6 20. Compare your results with the exact error. Repeat the computation but with T 5 650 6 40. Interpret your results. 4.10 Repeat Prob. 4.9 but for a copper sphere with radius 5 0.15 6 0.01 m, e 5 0.90 6 0.05, and T 5 550 6 20. 4.11 Recall that the velocity of the falling parachutist can be com- puted by [Eq. (1.10)],

y(t) 5 gm

c (1 2 e2 (cym)t)

Use a fi rst-order error analysis to estimate the error of v at t 5 6, if g 5 9.81 and m 5 50 but c 5 12.5 6 1.5. 4.12 Repeat Prob. 4.11 with g 5 9.81, t 5 6, c 5 12.5 6 1.5, and m 5 50 6 2. 4.13 Evaluate and interpret the condition numbers for (a) f (x) 5 1Zx 2 1Z 1 1 for x 5 1.00001 (b) f(x) 5 e2x for x 5 10 (c) f (x) 5 2x2 1 1 2 x for x 5 300

(d) f (x) 5 e2x 2 1

x for x 5 0.001

(e) f (x) 5 sin x

1 1 cos x for x 5 1.0001p

4.14 Employing ideas from Sec. 4.2, derive the relationships from Table 4.3. 4.15 Prove that Eq. (4.4) is exact for all values of x if f(x) 5 ax2 1 bx 1 c.

cha9792x_ch04_081-109.indd Page 108 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 108 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 109

4.16 Manning’s formula for a rectangular channel can be written as

Q 5 1 n

(BH)5y3

(B 1 2H)2y3 1S

where Q 5 fl ow (m3/s), n 5 a roughness coeffi cient, B 5 width (m), H 5 depth (m), and S 5 slope. You are applying this formula to a stream where you know that the width 5 20 m and the depth 5 0.3 m. Unfortunately, you know the roughness and the slope to only a 6 10% precision. That is, you know that the roughness is about 0.03 with a range from 0.027 to 0.033 and the slope is 0.0003 with a range from 0.00027 to 0.00033. Use a fi rst-order error analysis to determine the sensitivity of the fl ow prediction to each of these two factors. Which one should you attempt to measure with more precision? 4.17 If Z x Z , 1, it is known that

1

1 2 x 5 1 1 x 1 x2 1 x3 1 p

Repeat Prob. 4.1 for this series for x 5 0.1. 4.18 A missile leaves the ground with an initial velocity y0 form- ing an angle f0 with the vertical as shown in Fig. P4.18. The maxi-

mum desired altitude is aR where R is the radius of the earth. The laws of mechanics can be used to show that

sin f0 5 (1 1 a)B 1 2

a

1 1 a aye y0 b2

where ye 5 the escape velocity of the missile. It is desired to fi re the missile and reach the design maximum altitude within an accuracy of 62%. Determine the range of values for f0 if yeyy0 5 2 and a 5 0.25. 4.19 To calculate a planet’s space coordinates, we have to solve the function

f (x) 5 x 2 1 2 0.5 sin x

Let the base point be a 5 xi 5 py2 on the interval [0, p]. Determine the highest-order Taylor series expansion resulting in a maximum error of 0.015 on the specifi ed interval. The error is equal to the absolute value of the difference between the given function and the specifi c Taylor series expansion. (Hint: Solve graphically.) 4.20 Consider the function f(x) 5 x3 2 2x 1 4 on the interval [22, 2] with h 5 0.25. Use the forward, backward, and centered fi nite differ- ence approximations for the fi rst and second derivatives so as to graphically illustrate which approximation is most accurate. Graph all three fi rst derivative fi nite difference approximations along with the theoretical, and do the same for the second derivative as well. 4.21 Derive Eq. (4.31). 4.22 Repeat Example 4.8, but for f(x) 5 cos(x) at x 5 py6. 4.23 Repeat Example 4.8, but for the forward divided difference (Eq. 4.17). 4.24 Develop a well-structured program to compute the Maclaurin series expansion for the cosine function as described in Prob. 4.2. The function should have the following features:

• Iterate until the relative error falls below a stopping criterion (es) or exceeds a maximum number of iterations (maxit). Allow the user to specify values for these parameters.

• Include default values of es (5 0.000001) and maxit (5 100) in the event that they are not specifi ed by the user.

• Return the estimate of cos(x), the approximate relative error, the number of iterations, and the true relative error (that you can calculate based on the built-in cosine function).

FIGURE P4.18

R

v0

�0

cha9792x_ch04_081-109.indd Page 109 09/10/13 7:34 PM F-468 cha9792x_ch04_081-109.indd Page 109 09/10/13 7:34 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

110 EPILOGUE: PART ONE

110

EPILOGUE: PART ONE

PT1.4 TRADE-OFFS

Numerical methods are scientifi c in the sense that they represent systematic techniques for solving mathematical problems. However, there is a certain degree of art, subjective judgment, and compromise associated with their effective use in engineering practice. For each problem, you may be confronted with several alternative numerical methods and many different types of computers. Thus, the elegance and effi ciency of different approaches to problems is highly individualistic and correlated with your ability to choose wisely among options. Unfortunately, as with any intuitive process, the factors infl uencing this choice are diffi cult to communicate. Only by experience can these skills be fully comprehended and honed. However, because these skills play such a prominent role in the effective implementation of the methods, we have included this section as an introduction to some of the trade-offs that you must consider when selecting a numerical method and the tools for implementing the method. It is hoped that the discussion that follows will infl uence your orientation when approaching subsequent material. Also, it is hoped that you will refer back to this material when you are confronted with choices and trade-offs in the remainder of the book.

1. Type of Mathematical Problem. As delineated previously in Fig. PT1.2, several types of mathematical problems are discussed in this book: (a) Roots of equations. (b) Systems of simultaneous linear algebraic equations. (c) Optimization. (d) Curve fi tting. (e) Numerical integration. (f) Ordinary differential equations. (g) Partial differential equations.

You will probably be introduced to the applied aspects of numerical methods by confront- ing a problem in one of the above areas. Numerical methods will be required because the problem cannot be solved effi ciently using analytical techniques. You should be cognizant of the fact that your professional activities will eventually involve problems in all the above areas. Thus, the study of numerical methods and the selection of automatic computation equipment should, at the minimum, consider these basic types of problems. More advanced problems may require capabilities of handling areas such as functional approximation, integral equations, etc. These areas typically demand greater computation power or advanced methods not covered in this text. Other references such as Carnahan, Luther, and Wilkes (1969); Hamming (1973); Ralston and Rabinowitz (1978); Burden and Faires (2005); and Moler (2004) should be consulted for problems beyond the scope of this book. In addition, at the end of each part of this text, we include a brief summary

cha9792x_ep01_110-115.indd Page 110 06/10/13 5:52 PM F-468 cha9792x_ep01_110-115.indd Page 110 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT1.4 TRADE-OFFS 111

and references for advanced methods to provide you with avenues for pursuing further studies of numerical methods. 2. Type, Availability, Precision, Cost, and Speed of Computer. You may have the option

of working with a variety of computation tools. These range from pocket calculators to large mainframe computers. Of course, any of the tools can be used to implement any numerical method (including simple paper and pencil). It is usually not a question of ultimate capability but rather of cost, convenience, speed, dependability, repeatability, and precision. Although each of the tools will continue to have utility, the recent rapid advances in the performance of personal computers have already had a major impact on the engineering profession. We expect this revolution will spread as technological improvements continue because personal computers offer an excellent compromise in convenience, cost, precision, speed, and storage capacity. Furthermore, they can be readily applied to most practical engineering problems.

3. Program Development Cost versus Software Cost versus Run-Time Cost. Once the types of mathematical problems to be solved have been identifi ed and the computer system has been selected, it is appropriate to consider software and run-time costs. Software development may represent a substantial effort in many engineering projects and may therefore be a signifi cant cost. In this regard, it is particularly important that you be very well acquainted with the theoretical and practical aspects of the relevant numerical methods. In addition, you should be familiar with professionally developed software. Low-cost software is widely available to implement numerical methods that may be readily adapted to a broad variety of problems.

4. Characteristics of the Numerical Method. When computer hardware and software costs are high, or if computer availability is limited (for example, on some timeshare systems), it pays to choose carefully the numerical method to suit the situation. On the other hand, if the problem is still at the exploratory stage and computer access and cost are not problems, it may be appropriate for you to select a numerical method that always works but may not be the most computationally effi cient. The numerical methods available to solve any particular type of problem involve the types of trade- offs just discussed and others: (a) Number of Initial Guesses or Starting Points. Some of the numerical methods for

fi nding roots of equations or solving differential equations require the user to specify initial guesses or starting points. Simple methods usually require one value, whereas complicated methods may require more than one value. The advantages of complicated methods that are computationally effi cient may be offset by the requirement for multiple starting points. You must use your experience and judgment to assess the trade-offs for each particular problem.

(b) Rate of Convergence. Certain numerical methods converge more rapidly than others. However, this rapid convergence may require more refi ned initial guesses and more complex programming than a method with slower convergence. Again, you must use your judgment in selecting a method. Faster is not always better.

(c) Stability. Some numerical methods for fi nding roots of equations or solutions for systems of linear equations may diverge rather than converge on the correct answer for certain problems. Why would you tolerate this possibility when confronted with design or planning problems? The answer is that these methods may be highly effi cient when they work. Thus, trade-offs again emerge. You must decide

cha9792x_ep01_110-115.indd Page 111 06/10/13 5:52 PM F-468 cha9792x_ep01_110-115.indd Page 111 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

112 EPILOGUE: PART ONE

if your problem requirements justify the effort needed to apply a method that may not always converge.

(d) Accuracy and Precision. Some numerical methods are simply more accurate or precise than others. Good examples are the various equations available for numerical integration. Usually, the performance of low-accuracy methods can be improved by decreasing the step size or increasing the number of applications over a given interval. Is it better to use a low-accuracy method with small step sizes or a high-accuracy method with large step sizes? This question must be addressed on a case-by-case basis taking into consideration the additional factors such as cost and ease of programming. In addition, you must also be concerned with round-off errors when you are using multiple applications of low-accuracy methods and when the number of computations becomes large. Here the number of signifi cant fi gures handled by the computer may be the deciding factor.

(e) Breadth of Application. Some numerical methods can be applied to only a limited class of problems or to problems that satisfy certain mathematical restrictions. Other methods are not affected by such limitations. You must evaluate whether it is worth your effort to develop programs that employ techniques that are appropriate for only a limited number of problems. The fact that such techniques may be widely used suggests that they have advantages that will often outweigh their disadvantages. Obviously, trade-offs are occurring.

(f) Special Requirements. Some numerical techniques attempt to increase accuracy and rate of convergence using additional or special information. An example would be to use estimated or theoretical values of errors to improve accuracy. However, these improvements are generally not achieved without some inconvenience in terms of added computing costs or increased program complexity.

(g) Programming Effort Required. Efforts to improve rates of convergence, stability, and accuracy can be creative and ingenious. When improvements can be made without increasing the programming complexity, they may be considered elegant and will probably fi nd immediate use in the engineering profession. However, if they require more complicated programs, you are again faced with a trade-off situation that may or may not favor the new method.

It is clear that the above discussion concerning a choice of numerical methods reduces to one of cost and accuracy. The costs are those involved with computer time and program development. Appropriate accuracy is a question of professional judg- ment and ethics.

5. Mathematical Behavior of the Function, Equation, or Data. In selecting a particular numerical method, type of computer, and type of software, you must consider the complexity of your functions, equations, or data. Simple equations and smooth data may be appropriately handled by simple numerical algorithms and inexpensive computers. The opposite is true for complicated equations and data exhibiting discontinuities.

6. Ease of Application (User-Friendly?). Some numerical methods are easy to apply; others are diffi cult. This may be a consideration when choosing one method over

cha9792x_ep01_110-115.indd Page 112 06/10/13 5:52 PM F-468 cha9792x_ep01_110-115.indd Page 112 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT1.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 113

another. This same idea applies to decisions regarding program development costs versus professionally developed software. It may take considerable effort to convert a diffi cult program to one that is user-friendly. Ways to do this were introduced in Chap. 2 and are elaborated throughout the book.

7. Maintenance. Programs for solving engineering problems require maintenance because during application, diffi culties invariably occur. Maintenance may require changing the program code or expanding the documentation. Simple programs and numerical algorithms are simpler to maintain.

The chapters that follow involve the development of various types of numerical methods for various types of mathematical problems. Several alternative methods will be given in each chapter. These various methods (rather than a single method chosen by the au- thors) are presented because there is no single “best” method. There is no best method because there are many trade-offs that must be considered when applying the methods to practical problems. A table that highlights the trade-offs involved in each method will be found at the end of each part of the book. This table should assist you in selecting the appropriate numerical procedure for your particular problem context.

PT1.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT1.2 summarizes important information that was presented in Part One. The table can be consulted to quickly access important relationships and formulas. The epilogue of each part of the book will contain such a summary.

PT1.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

The epilogue of each part of the book will also include a section designed to facilitate and encourage further studies of numerical methods. This section will reference other books on the subject as well as material related to more advanced methods. 1 To extend the background provided in Part One, numerous manuals on computer programming are available. It would be diffi cult to reference all the excellent books and manuals pertaining to specifi c languages and computers. In addition, you probably already have material from your previous exposure to programming. However, if this is your fi rst experience with computers, your instructor and fellow students should also be able to advise you regarding good reference books for the machines and languages available at your school. As for error analysis, any good introductory calculus book will include supplemen- tary material related to subjects such as the Taylor series expansion. Texts by Swokowski (1979), Thomas and Finney (1979), and Simmons (1985) provide very readable discus- sions of these subjects. In addition, Taylor (1982) presents a nice introduction to error analysis. Finally, although we hope that our book serves you well, it is always good to con- sult other sources when trying to master a new subject. Burden and Faires (2005); Ralston

1 Books are referenced only by author here; a complete bibliography will be found at the back of this text.

cha9792x_ep01_110-115.indd Page 113 06/10/13 5:52 PM F-468 cha9792x_ep01_110-115.indd Page 113 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

114 EPILOGUE: PART ONE

TABLE PT1.2 Summary of important information presented in Part One.

Error Defi nitions True error E t 5 true value 2 approximation

True percent relative error et 5 true value 2 approximation

true value 100%

Approximate percent relative error ea 5 present approximation 2 previous approximation

present approximation 100%

Stopping criterion Terminate computation when ea , es where e s is the desired percent relative error

Taylor Series Taylor series expansion

f (xi11) 5 f (xi) 1 f' (xi)h 1 f'' (xi) 2!

h2

1 f''' (xi)

3! h3 1 p 1

f (n) (xi) n!

hn 1 Rn

where

Remainder Rn 5 f (n11)(j) (n 1 1)!

hn11

or

Rn 5 O(hn11)

Numerical Differentiation First forward fi nite divided difference

f '(xi) 5 f (xi11) 2 f (xi)

h 1 O(h)

(Other divided differences are summarized in Chaps. 4 and 23.)

Error Propagation For n independent variables x 1 , x 2 , . . . , x n having errors ¢ x̃1, ¢x̃2 , p ,¢x̃n, the error in the function f can be estimated via

¢f 5 ` 0f 0x1 ` ¢x̃1 1 ` 0f0x2 ` ¢x̃2 1 p 1 `

0f 0xn ` ¢x̃n

and Rabinowitz (1978); Hoffman (1992); and Carnahan, Luther, and Wilkes (1969) pro- vide comprehensive discussions of most numerical methods. Other enjoyable books on the subject are Gerald and Wheatley (2004), and Cheney and Kincaid (2008). In addition, Press et al. (2007) include algorithms to implement a variety of methods, and Moler (2004) and Chapra (2007) are devoted to numerical methods with MATLAB software.

cha9792x_ep01_110-115.indd Page 114 06/10/13 5:52 PM F-468 cha9792x_ep01_110-115.indd Page 114 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

This page intentionally left blank

PART TWO

cha9792x_p02_116-122.indd Page 116 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 116 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

117

PT2.1 MOTIVATION

Years ago, you learned to use the quadratic formula

x 5 2b 6 2b2 2 4ac

2a (PT2.1)

to solve

f (x) 5 ax2 1 bx 1 c 5 0 (PT2.2)

The values calculated with Eq. (PT2.1) are called the “roots” of Eq. (PT2.2). They rep- resent the values of x that make Eq. (PT2.2) equal to zero. Thus, we can defi ne the root of an equation as the value of x that makes f ( x ) 5 0. For this reason, roots are sometimes called the zeros of the equation. Although the quadratic formula is handy for solving Eq. (PT2.2), there are many other functions for which the root cannot be determined so easily. For these cases, the numerical methods described in Chaps. 5, 6, and 7 provide effi cient means to obtain the answer.

PT2.1.1 Noncomputer Methods for Determining Roots

Before the advent of digital computers, there were several ways to solve for roots of algebraic and transcendental equations. For some cases, the roots could be obtained by direct methods, as was done with Eq. (PT2.1). Although there were equations like this that could be solved directly, there were many more that could not. For example, even an apparently simple function such as f ( x ) 5 e 2 x 2 x cannot be solved analytically. In such instances, the only alternative is an approximate solution technique. One method to obtain an approximate solution is to plot the function and determine where it crosses the x axis. This point, which represents the x value for which f ( x ) 5 0, is the root. Graphical techniques are discussed at the beginning of Chaps. 5 and 6. Although graphical methods are useful for obtaining rough estimates of roots, they are limited because of their lack of precision. An alternative approach is to use trial and error. This “technique” consists of guessing a value of x and evaluating whether f ( x ) is zero. If not (as is almost always the case), another guess is made, and f ( x ) is again evaluated to determine whether the new value provides a better estimate of the root. The process is repeated until a guess is obtained that results in an f ( x ) that is close to zero. Such haphazard methods are obviously ineffi cient and inadequate for the require- ments of engineering practice. The techniques described in Part Two represent alterna- tives that are also approximate but employ systematic strategies to home in on the true root. As elaborated on in the following pages, the combination of these systematic meth- ods and computers makes the solution of most applied roots-of-equations problems a simple and effi cient task.

ROOTS OF EQUATIONS

cha9792x_p02_116-122.indd Page 117 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 117 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

118 ROOTS OF EQUATIONS

PT2.1.2 Roots of Equations and Engineering Practice

Although they arise in other problem contexts, roots of equations frequently occur in the area of engineering design. Table PT2.1 lists several fundamental principles that are routinely used in design work. As introduced in Chap. 1, mathematical equations or models derived from these principles are employed to predict dependent variables as a function of independent variables, forcing functions, and parameters. Note that in each case, the dependent variables refl ect the state or performance of the system, whereas the parameters represent its properties or composition. An example of such a model is the equation, derived from Newton’s second law, used in Chap. 1 for the parachutist’s velocity:

y 5 gm

c (1 2 e2 (cym)t) (PT2.3)

where velocity y 5 the dependent variable, time t 5 the independent variable, the grav- itational constant g 5 the forcing function, and the drag coeffi cient c and mass m 5 parameters. If the parameters are known, Eq. (PT2.3) can be used to predict the parachut- ist’s velocity as a function of time. Such computations can be performed directly because y is expressed explicitly as a function of time. That is, it is isolated on one side of the equal sign.

TABLE PT2.1 Fundamental principles used in engineering design problems.

Fundamental Dependent Independent Parameters Principle Variable Variable

Heat balance Temperature Time and Thermal properties position of material and geometry of system Mass balance Concentration or Time and Chemical behavior quantity of mass position of material, mass transfer coeffi cients, and geometry of system Force balance Magnitude and Time and Strength of material, direction of forces position structural properties, and geometry of system Energy balance Changes in the kinetic- Time and Thermal properties, and potential-energy position mass of material, states of the system and system geometry Newton’s laws Acceleration, velocity, Time and Mass of material, of motion or location position system geometry, and dissipative parameters such as friction or drag Kirchhoff’s laws Currents and voltages Time Electrical properties in electric circuits of systems such as resistance, capacitance, and inductance

cha9792x_p02_116-122.indd Page 118 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 118 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT2.2 MATHEMATICAL BACKGROUND 119

However, suppose we had to determine the drag coeffi cient for a parachutist of a given mass to attain a prescribed velocity in a set time period. Although Eq. (PT2.3) provides a mathematical representation of the interrelationship among the model vari- ables and parameters, it cannot be solved explicitly for the drag coeffi cient. Try it. There is no way to rearrange the equation so that c is isolated on one side of the equal sign. In such cases, c is said to be implicit. This represents a real dilemma, because many engineering design problems involve specifying the properties or composition of a system (as represented by its parameters) to ensure that it performs in a desired manner (as represented by its variables). Thus, these problems often require the determination of implicit parameters. The solution to the dilemma is provided by numerical methods for roots of equations. To solve the problem using numerical methods, it is conventional to reexpress Eq. (PT2.3). This is done by subtracting the dependent variable y from both sides of the equation to give

f (c) 5 gm

c (1 2 e2 (cym)t) 2 y (PT2.4)

The value of c that makes f ( c ) 5 0 is, therefore, the root of the equation. This value also represents the drag coeffi cient that solves the design problem. Part Two of this book deals with a variety of numerical and graphical methods for deter- mining roots of relationships such as Eq. (PT2.4). These techniques can be applied to engi- neering design problems that are based on the fundamental principles outlined in Table PT2.1 as well as to many other problems confronted routinely in engineering practice.

PT2.2 MATHEMATICAL BACKGROUND

For most of the subject areas in this book, there is usually some prerequisite mathematical background needed to successfully master the topic. For example, the concepts of error estimation and the Taylor series expansion discussed in Chaps. 3 and 4 have direct relevance to our discussion of roots of equations. Additionally, prior to this point we have mentioned the terms “algebraic” and “transcendental” equations. It might be helpful to formally defi ne these terms and discuss how they relate to the scope of this part of the book. By defi nition, a function given by y 5 f ( x ) is algebraic if it can be expressed in the form

fn y n 1 fn21 y

n21 1 p 1 f1 y 1 f0 5 0 (PT2.5)

where f i 5 an i th-order polynomial in x. Polynomials are a simple class of algebraic functions that are represented generally by

fn(x) 5 a0 1 a1x 1 a2x 2 1 p 1 an x n (PT2.6)

where n 5 the order of the polynomial and the a ’s 5 constants. Some specifi c examples are

f2(x) 5 1 2 2.37x 1 7.5x  2 (PT2.7)

and

f6(x) 5 5x 2 2 x3 1 7x6 (PT2.8)

cha9792x_p02_116-122.indd Page 119 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 119 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

120 ROOTS OF EQUATIONS

A transcendental function is one that is nonalgebraic. These include trigonometric, exponential, logarithmic, and other, less familiar, functions. Examples are

f (x) 5 ln x 2 2 1 (PT2.9)

and

f(x) 5 e20.2x sin (3x 2 0.5) (PT2.10)

Roots of equations may be either real or complex. Although there are cases where com- plex roots of nonpolynomials are of interest, such situations are less common than for polynomials. As a consequence, the standard methods for locating roots typically fall into two somewhat related but primarily distinct problem areas:

1. The determination of the real roots of algebraic and transcendental equations. These techniques are usually designed to determine the value of a single real root on the basis of foreknowledge of its approximate location.

2. The determination of all real and complex roots of polynomials. These methods are specifi cally designed for polynomials. They systematically determine all the roots of the polynomial rather than determining a single real root given an approximate location.

In this book we discuss both. Chapters 5 and 6 are devoted to the fi rst category. Chapter 7 deals with polynomials.

PT2.3 ORIENTATION

Some orientation is helpful before proceeding to the numerical methods for determining roots of equations. The following is intended to give you an overview of the material in Part Two. In addition, some objectives have been included to help you focus your efforts when studying the material.

PT2.3.1 Scope and Preview

Figure PT2.1 is a schematic representation of the organization of Part Two. Examine this fi gure carefully, starting at the top and working clockwise. After the present introduction, Chap. 5 is devoted to bracketing methods for fi nding roots. These methods start with guesses that bracket, or contain, the root and then sys- tematically reduce the width of the bracket. Two specifi c methods are covered: bisection and false position. Graphical methods are used to provide visual insight into the tech- niques. Error formulations are developed to help you determine how much computational effort is required to estimate the root to a prespecifi ed level of precision. Chapter 6 covers open methods. These methods also involve systematic trial-and- error iterations but do not require that the initial guesses bracket the root. We will dis- cover that these methods are usually more computationally effi cient than bracketing methods but that they do not always work. One-point iteration, Newton- Raphson , and secant methods are described. Graphical methods are used to provide geometric insight into cases where the open methods do not work. Formulas are developed that provide an idea of how fast open methods home in on the root. An advanced approach, Brent’s method , that combines the reliability of bracketing with the speed of open methods is

cha9792x_p02_116-122.indd Page 120 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 120 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT2.3 ORIENTATION 121

described. In addition, an approach to extend the Newton-Raphson method to systems of nonlinear equations is explained. Chapter 7 is devoted to fi nding the roots of polynomials. After background sections on polynomials, the use of conventional methods (in particular the open methods from Chap. 6) are discussed. Then two special methods for locating polynomial roots are

CHAPTER 5 Bracketing Methods

PART 2 Roots

of Equations

CHAPTER 7 Roots

of Polynomials

CHAPTER 8 Engineering Case Studies

EPILOGUE

6.6 Nonlinear systems

6.5 Multiple

roots

6.4 Brent’s method

6.3 Secant

6.2 Newton- Raphson

6.1 Fixed-point

iteration

PT 2.2 Mathematical background

PT 2.6 Advanced methods

PT 2.5 Important formulas

8.4 Mechanical engineering

8.3 Electrical

engineering

8.2 Civil

engineering

8.1 Chemical

engineering

7.7 Software packages

7.6 Other

methods

7.1 Polynomials in

engineering

7.2 Computing with

polynomials

7.4 Muller's method

7.5 Bairstow's

method 7.3

Conventional methods

PT 2.4 Trade-offs

PT 2.3 Orientation

PT 2.1 Motivation

5.2 Bisection

5.3 False

position

5.4 Incremental

searches

5.1 Graphical methods

..

CHAPTER 6 Open

Methods

FIGURE PT2.1 Schematic of the organization of the material in Part Two: Roots of Equations.

cha9792x_p02_116-122.indd Page 121 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 121 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

122 ROOTS OF EQUATIONS

described: Müller’s and Bairstow’s methods. The chapter ends with information related to fi nding roots with Excel, MATLAB software, and Mathcad. Chapter 8 extends the above concepts to actual engineering problems. Engineering case studies are used to illustrate the strengths and weaknesses of each method and to provide insight into the application of the techniques in professional practice. The applications also highlight the trade-offs (as discussed in Part One) associated with the various methods. An epilogue is included at the end of Part Two. It contains a detailed comparison of the methods discussed in Chaps. 5, 6, and 7. This comparison includes a description of trade-offs related to the proper use of each technique. This section also provides a summary of important formulas, along with references for some numerical methods that are beyond the scope of this text.

PT2.3.2 Goals and Objectives

Study Objectives. After completing Part Two, you should have suffi cient information to successfully approach a wide variety of engineering problems dealing with roots of equations. In general, you should have mastered the techniques, have learned to assess their reliability, and be capable of choosing the best method (or methods) for any par- ticular problem. In addition to these general goals, the specifi c concepts in Table PT2.2 should be assimilated for a comprehensive understanding of the material in Part Two.

Computer Objectives. The book provides you with software and simple computer algo- rithms to implement the techniques discussed in Part Two. All have utility as learning tools. Pseudocodes for several methods are also supplied directly in the text. This informa- tion will allow you to expand your software library to include programs that are more effi cient than the bisection method. For example, you may also want to have your own software for the false-position, Newton-Raphson, and secant techniques, which are often more effi cient than the bisection method. Finally, packages such as Excel, MATLAB, and Mathcad have powerful capabilities for locating roots. You can use this part of the book to become familiar with these capabilities.

TABLE PT2.2 Specifi c study objectives for Part Two.

1. Understand the graphical interpretation of a root 2. Know the graphical interpretation of the false-position method and why it is usually superior to the

bisection method 3. Understand the difference between bracketing and open methods for root location 4. Understand the concepts of convergence and divergence; use the two-curve graphical method to

provide a visual manifestation of the concepts 5. Know why bracketing methods always converge, whereas open methods may sometimes diverge 6. Realize that convergence of open methods is more likely if the initial guess is close to the true root 7. Understand the concepts of linear and quadratic convergence and their implications for the

effi ciencies of the fi xed-point-iteration and Newton-Raphson methods 8. Know the fundamental difference between the false-position and secant methods and how it relates

to convergence 9. Understand how Brent’s method combines the reliability of bisection with the speed of open methods 10. Understand the problems posed by multiple roots and the modifi cations available to mitigate them 11. Know how to extend the single-equation Newton-Raphson approach to solve systems of nonlinear

equations

cha9792x_p02_116-122.indd Page 122 15/10/13 7:17 PM f-479 cha9792x_p02_116-122.indd Page 122 15/10/13 7:17 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

123

5

Bracketing Methods

This chapter on roots of equations deals with methods that exploit the fact that a function typically changes sign in the vicinity of a root. These techniques are called bracketing methods because two initial guesses for the root are required. As the name implies, these guesses must “bracket,” or be on either side of, the root. The particular methods described herein employ different strategies to systematically reduce the width of the bracket and, hence, home in on the correct answer. As a prelude to these techniques, we will briefl y discuss graphical methods for depicting functions and their roots. Beyond their utility for providing rough guesses, graphical techniques are also useful for visualizing the properties of the functions and the behavior of the various numerical methods.

5.1 GRAPHICAL METHODS

A simple method for obtaining an estimate of the root of the equation f(x) 5 0 is to make a plot of the function and observe where it crosses the x axis. This point, which represents the x value for which f(x) 5 0, provides a rough approximation of the root.

EXAMPLE 5.1 The Graphical Approach

Problem Statement. Use the graphical approach to determine the drag coeffi cient c needed for a parachutist of mass m 5 68.1 kg to have a velocity of 40 m/s after free- falling for time t 5 10 s. Note: The acceleration due to gravity is 9.81 m/s2.

Solution. This problem can be solved by determining the root of Eq. (PT2.4) using the parameters t 5 10, g 5 9.81, y 5 40, and m 5 68.1:

f(c) 5 9.81(68.1)

c (1 2 e2(cy68.1)10) 2 40

or

f(c) 5 668.06

c (1 2 e20.146843c) 2 40 (E5.1.1)

Various values of c can be substituted into the right-hand side of this equation to compute

C H A P T E R 5

cha9792x_ch05_123-144.indd Page 123 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 123 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

124 BRACKETING METHODS

These points are plotted in Fig. 5.1. The resulting curve crosses the c axis between 12 and 16. Visual inspection of the plot provides a rough estimate of the root of 14.75. The valid- ity of the graphical estimate can be checked by substituting it into Eq. (E5.1.1) to yield

f(14.75) 5 668.06

14.75 (1 2 e20.146843(14.75)) 2 40 5 0.100

which is close to zero. It can also be checked by substituting it into Eq. (PT2.3) along with the parameter values from this example to give

y 5 9.81(68.1)

14.75 (1 2 e2 (14.75y68.1)10) 5 40.100

which is very close to the desired fall velocity of 40 m/s.

c f (c)

4 34.190 8 17.712 12 6.114 16 22.230 20 28.368

FIGURE 5.1 The graphical approach for determining the roots of an equation.

20

Root

12840

20

40

f (c)

c

–10

cha9792x_ch05_123-144.indd Page 124 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 124 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.1 GRAPHICAL METHODS 125

Graphical techniques are of limited practical value because they are not precise. However, graphical methods can be utilized to obtain rough estimates of roots. These estimates can be employed as starting guesses for numerical methods discussed in this and the next chapter. Aside from providing rough estimates of the root, graphical interpretations are im- portant tools for understanding the properties of the functions and anticipating the pitfalls of the numerical methods. For example, Fig. 5.2 shows a number of ways in which roots can occur (or be absent) in an interval prescribed by a lower bound xl and an upper bound xu. Figure 5.2b depicts the case where a single root is bracketed by negative and positive values of f(x). However, Fig. 5.2d, where f(xl) and f(xu) are also on opposite sides of the x axis, shows three roots occurring within the interval. In general, if f(xl) and f(xu) have opposite signs, there are an odd number of roots in the interval. As indi- cated by Fig. 5.2a and c, if f(xl) and f(xu) have the same sign, there are either no roots or an even number of roots between the values. Although these generalizations are usually true, there are cases where they do not hold. For example, functions that are tangential to the x axis (Fig. 5.3a) and discontinu- ous functions (Fig. 5.3b) can violate these principles. An example of a function that is tangential to the axis is the cubic equation f(x) 5 (x 2 2)(x 2 2)(x 2 4). Notice that x 5 2 makes two terms in this polynomial equal to zero. Mathematically, x 5 2 is called a multiple root. At the end of Chap. 6, we will present techniques that are expressly designed to locate multiple roots. The existence of cases of the type depicted in Fig. 5.3 makes it diffi cult to develop general computer algorithms guaranteed to locate all the roots in an interval. However, when used in conjunction with graphical approaches, the methods described in the

FIGURE 5.2 Illustration of a number of general ways that a root may occur in an interval prescribed by a lower bound xl and an upper bound xu. Parts (a) and (c) indicate that if both f(xl) and f(xu) have the same sign, either there will be no roots or there will be an even number of roots within the interval. Parts (b) and (d) indicate that if the function has different signs at the end points, there will be an odd number of roots in the interval.

f (x)

x

f (x)

x

f (x)

x

f (x)

x

(a)

(b)

(c)

(d) xl xu

FIGURE 5.3 Illustration of some exceptions to the general cases depicted in Fig. 5.2. (a) Multiple root that occurs when the function is tangen- tial to the x axis. For this case, although the end points are of op- posite signs, there are an even number of axis intersections for the interval. (b) Discontinuous function where end points of oppo- site sign bracket an even number of roots. Special strategies are required for determining the roots for these cases.

f (x)

x

f (x)

x

(a)

(b) xl xu

cha9792x_ch05_123-144.indd Page 125 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 125 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

126 BRACKETING METHODS

following sections are extremely useful for solving many roots of equations problems confronted routinely by engineers and applied mathematicians.

EXAMPLE 5.2 Use of Computer Graphics to Locate Roots

Problem Statement. Computer graphics can expedite and improve your efforts to locate roots of equations. The function

f(x) 5 sin 10x 1 cos 3x

has several roots over the range x 5 0 to x 5 5. Use computer graphics to gain insight into the behavior of this function.

Solution. Packages such as Excel and MATLAB software can be used to generate plots. Figure 5.4a is a plot of f(x) from x 5 0 to x 5 5. This plot suggests the presence of several roots, including a possible double root at about x 5 4.2 where f(x) appears to be

.15

0Y

4.2 4.25 X

4.3 – .15

(c)

FIGURE 5.4 The progressive enlargement of f(x) 5 sin 10x 1 cos 3x by the computer. Such interactive graphics permits the analyst to determine that two distinct roots exist between x 5 4.2 and x 5 4.3.

5

2

0Y

0 2.5 X

– 2

2

0Y

3 4 X

5 – 2

(a) (b)

cha9792x_ch05_123-144.indd Page 126 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 126 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.2 THE BISECTION METHOD 127

tangent to the x axis. A more detailed picture of the behavior of f(x) is obtained by chang- ing the plotting range from x 5 3 to x 5 5, as shown in Fig. 5.4b. Finally, in Fig. 5.4c, the vertical scale is narrowed further to f(x) 5 20.15 to f(x) 5 0.15 and the horizontal scale is narrowed to x 5 4.2 to x 5 4.3. This plot shows clearly that a double root does not exist in this region and that in fact there are two distinct roots at about x 5 4.23 and x 5 4.26. Computer graphics will have great utility in your studies of numerical methods. This capability will also fi nd many other applications in your other classes and professional activities as well.

FIGURE 5.5 Step 1: Choose lower xl and upper xu guesses for the root such that the function changes sign

over the interval. This can be checked by ensuring that f(xl)f(xu) , 0. Step 2: An estimate of the root xr is determined by

xr 5 xl 1 xu

2

Step 3: Make the following evaluations to determine in which subinterval the root lies: (a) If f(xl)f(xr) , 0, the root lies in the lower subinterval. Therefore, set xu 5 xr and return

to step 2. (b) If f(xl)f(xr) . 0, the root lies in the upper subinterval. Therefore, set xl 5 xr and return

to step 2. (c) If f(xl)f(xr) 5 0, the root equals xr; terminate the computation.

5.2 THE BISECTION METHOD

When applying the graphical technique in Example 5.1, you have observed (Fig. 5.1) that f(x) changed sign on opposite sides of the root. In general, if f(x) is real and con- tinuous in the interval from xl to xu and f(xl) and f(xu) have opposite signs, that is,

f(xl) f(xu) , 0 (5.1)

then there is at least one real root between xl and xu. Incremental search methods capitalize on this observation by locating an interval where the function changes sign. Then the location of the sign change (and consequently, the root) is identifi ed more precisely by dividing the interval into a number of subinter- vals. Each of these subintervals is searched to locate the sign change. The process is repeated and the root estimate refi ned by dividing the subintervals into fi ner increments. We will return to the general topic of incremental searches in Sec. 5.4. The bisection method, which is alternatively called binary chopping, interval halving, or Bolzano’s method, is one type of incremental search method in which the interval is always divided in half. If a function changes sign over an interval, the function value at the midpoint is evaluated. The location of the root is then determined as lying at the midpoint of the subinterval within which the sign change occurs. The process is repeated to obtain refi ned estimates. A simple algorithm for the bisection calculation is listed in Fig. 5.5, and a graphical depiction of the method is provided in Fig. 5.6. The following example goes through the actual computations involved in the method.

cha9792x_ch05_123-144.indd Page 127 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 127 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

128 BRACKETING METHODS

EXAMPLE 5.3 Bisection

Problem Statement. Use bisection to solve the same problem approached graphically in Example 5.1.

Solution. The fi rst step in bisection is to guess two values of the unknown (in the present problem, c) that give values for f(c) with different signs. From Fig. 5.1, we can see that the function changes sign between values of 12 and 16. Therefore, the initial estimate of the root xr lies at the midpoint of the interval

xr 5 12 1 16

2 5 14

This estimate represents a true percent relative error of et 5 5.3% (note that the true value of the root is 14.8011). Next we compute the product of the function value at the lower bound and at the midpoint:

f(12) f(14) 5 6.114 (1.611) 5 9.850

which is greater than zero, and hence no sign change occurs between the lower bound and the midpoint. Consequently, the root must be located between 14 and 16. Therefore, we create a new interval by redefi ning the lower bound as 14 and determining a revised root estimate as

xr 5 14 1 16

2 5 15

which represents a true percent error of et 5 1.3%. The process can be repeated to obtain refi ned estimates. For example,

f(14) f(15) 5 1.611(20.384) 5 20.619

1612

14 16

15

14

FIGURE 5.6 A graphical depiction of the bisection method. This plot conforms to the fi rst three iterations from Example 5.3.

cha9792x_ch05_123-144.indd Page 128 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 128 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.2 THE BISECTION METHOD 129

Therefore, the root is between 14 and 15. The upper bound is redefi ned as 15, and the root estimate for the third iteration is calculated as

xr 5 14 1 15

2 5 14.5

which represents a percent relative error of et 5 2.0%. The method can be repeated until the result is accurate enough to satisfy your needs.

In the previous example, you may have noticed that the true error does not decrease with each iteration. However, the interval within which the root is located is halved with each step in the process. As discussed in the next section, the interval width provides an exact estimate of the upper bound of the error for the bisection method.

5.2.1 Termination Criteria and Error Estimates

We ended Example 5.3 with the statement that the method could be continued to obtain a refi ned estimate of the root. We must now develop an objective criterion for deciding when to terminate the method. An initial suggestion might be to end the calculation when the true error falls below some prespecifi ed level. For instance, in Example 5.3, the relative error dropped to 2.0 percent during the course of the computation. We might decide that we should terminate when the error drops below, say, 0.1 percent. This strategy is fl awed because the error estimates in the example were based on knowledge of the true root of the function. This would not be the case in an actual situation because there would be no point in using the method if we already knew the root. Therefore, we require an error estimate that is not contingent on foreknowledge of the root. As developed previously in Sec. 3.3, an approximate percent relative error ea can be calculated, as in [recall Eq. (3.5)]

ea 5 ` x new r 2 x

old r

xnewr ` 100% (5.2)

where xnewr is the root for the present iteration and x old r is the root from the previous it-

eration. The absolute value is used because we are usually concerned with the magnitude of ea rather than with its sign. When ea becomes less than a prespecifi ed stopping cri- terion es, the computation is terminated.

EXAMPLE 5.4 Error Estimates for Bisection

Problem Statement. Continue Example 5.3 until the approximate error falls below a stopping criterion of es 5 0.5%. Use Eq. (5.2) to compute the errors.

Solution. The results of the fi rst two iterations for Example 5.3 were 14 and 15. Sub- stituting these values into Eq. (5.2) yields

ZeaZ 5 ` 15 2 1415 ` 100% 5 6.667%

cha9792x_ch05_123-144.indd Page 129 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 129 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

130 BRACKETING METHODS

Recall that the true percent relative error for the root estimate of 15 was 1.3%. Therefore, ea is greater than et. This behavior is manifested for the other iterations:

Thus, after six iterations ea fi nally falls below es 5 0.5%, and the computation can be terminated. These results are summarized in Fig. 5.7. The “ragged” nature of the true error is due to the fact that, for bisection, the true root can lie anywhere within the bracketing interval. The true and approximate errors are far apart when the interval happens to be centered on the true root. They are close when the true root falls at either end of the interval.

Iteration xl xu xr Ea (%) et (%)

1 12 16 14 5.413 2 14 16 15 6.667 1.344 3 14 15 14.5 3.448 2.035 4 14.5 15 14.75 1.695 0.345 5 14.75 15 14.875 0.840 0.499 6 14.75 14.875 14.8125 0.422 0.077

FIGURE 5.7 Errors for the bisection method. True and estimated errors are plotted versus the number of iterations.

62 4

Iterations

P er

ce n

t re

la ti

ve e

rr o

r

0

0.1

1.0

True

Approximate

10

Although the approximate error does not provide an exact estimate of the true error, Fig. 5.7 suggests that ea captures the general downward trend of et. In addition, the plot exhibits the extremely attractive characteristic that ea is always greater than et. Thus,

cha9792x_ch05_123-144.indd Page 130 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 130 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.2 THE BISECTION METHOD 131

when ea falls below es, the computation could be terminated with confi dence that the root is known to be at least as accurate as the prespecifi ed acceptable level. Although it is always dangerous to draw general conclusions from a single example, it can be demonstrated that ea will always be greater than et for the bisection method. This is because each time an approximate root is located using bisection as xr 5 (xl 1 xu)y2, we know that the true root lies somewhere within an interval of (xu 2 xl)y2 5 Dxy2. Therefore, the root must lie within 6Dxy2 of our estimate (Fig. 5.8). For instance, when Example 5.3 was terminated, we could make the defi nitive statement that

xr 5 14.5 6 0.5

Because ¢xy2 5 xnewr 2 xoldr (Fig. 5.9), Eq. (5.2) provides an exact upper bound on the true error. For this bound to be exceeded, the true root would have to fall outside the bracketing interval, which, by defi nition, could never occur for the bisection method. As illustrated in a subsequent example (Example 5.6), other root-locating techniques do not always behave as nicely. Although bisection is generally slower than other methods,

FIGURE 5.8 Three ways in which the interval may bracket the root. In (a) the true value lies at the center of the interval, whereas in (b) and (c) the true value lies near the extreme. Notice that the dis- crepancy between the true value and the midpoint of the interval never exceeds half the interval length, or Dxy2.

(b)

(a)

(c)

�x /2

xl xr xu

xl xr xu

xl xr xu

�x /2

True root

FIGURE 5.9 Graphical depiction of why the error estimate for bisection (Dxy2) is equivalent to the root estimate for the present iteration (xnewr ) minus the root estimate for the previous iteration (xoldr ).

Previous iteration

�x /2

xoldr

xnewr

xnew – xoldr r

Present iteration

cha9792x_ch05_123-144.indd Page 131 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 131 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

132 BRACKETING METHODS

the neatness of its error analysis is certainly a positive aspect that could make it attrac- tive for certain engineering applications. Before proceeding to the computer program for bisection, we should note that the relationships (Fig. 5.9)

xnewr 2 x old r 5

xu 2 xl 2

and

xnewr 5 xl 1 xu

2

can be substituted into Eq. (5.2) to develop an alternative formulation for the approximate percent relative error

ea 5 ` xu 2 xlxu 1 xl ` 100% (5.3) This equation yields identical results to Eq. (5.2) for bisection. In addition, it allows us to calculate an error estimate on the basis of our initial guesses—that is, on our fi rst iteration. For instance, on the fi rst iteration of Example 5.2, an approximate error can be computed as

ea 5 ` 16 2 1216 1 12 ` 100% 5 14.29% Another benefi t of the bisection method is that the number of iterations required to attain an absolute error can be computed a priori—that is, before starting the iterations. This can be seen by recognizing that before starting the technique, the absolute error is

E0a 5 x 0 u 2 x

0 l 5 ¢x0

where the superscript designates the iteration. Hence, before starting the method, we are at the “zero iteration.” After the fi rst iteration, the error becomes

E1a 5 ¢x0

2

Because each succeeding iteration halves the error, a general formula relating the error and the number of iterations n is

Ena 5 ¢x0

2n (5.4)

If Ea,d is the desired error, this equation can be solved for

n 5 log(¢x0yEa,d)

log 2 5 log2 a¢x

0

Ea,d b (5.5)

Let us test the formula. For Example 5.4, the initial interval was Dx0 5 16 2 12 5 4. After six iterations, the absolute error was

Ea 5 Z14.875 2 14.75 Z

2 5 0.0625

cha9792x_ch05_123-144.indd Page 132 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 132 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.2 THE BISECTION METHOD 133

We can substitute these values into Eq. (5.5) to give

n 5 log(4y0.0625)

log 2 5 6

Thus, if we knew beforehand that an error of less than 0.0625 was acceptable, the for- mula tells us that six iterations would yield the desired result. Although we have emphasized the use of relative errors for obvious reasons, there will be cases where (usually through knowledge of the problem context) you will be able to specify an absolute error. For these cases, bisection along with Eq. (5.5) can provide a useful root-location algorithm. We will explore such applications in the end-of-chapter problems.

5.2.2 Bisection Algorithm

The algorithm in Fig. 5.5 can now be expanded to include the error check (Fig. 5.10). The algorithm employs user-defi ned functions to make root location and function evalu- ation more effi cient. In addition, an upper limit is placed on the number of iterations. Finally, an error check is included to avoid division by zero during the error evaluation. Such would be the case when the bracketing interval is centered on zero. For this situ- ation, Eq. (5.2) becomes infi nite. If this occurs, the program skips over the error evalu- ation for that iteration. The algorithm in Fig. 5.10 is not user-friendly; it is designed strictly to come up with the answer. In Prob. 5.14 at the end of this chapter, you will have the task of mak- ing it easier to use and understand.

FUNCTION Bisect(xl, xu, es, imax, xr, iter, ea) iter 5 0 DO xrold 5 xr xr 5 (xl 1 xu) / 2 iter 5 iter 1 1 IF xr ? 0 THEN ea 5 ABS((xr 2 xrold) / xr) * 100 END IF test 5 f(xl) * f(xr) IF test , 0 THEN xu 5 xr ELSE IF test . 0 THEN xl 5 xr ELSE ea 5 0 END IF IF ea , es OR iter $ imax EXIT END DO Bisect 5 xr END Bisect

FIGURE 5.10 Pseudocode for function to implement bisection.

cha9792x_ch05_123-144.indd Page 133 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 133 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

134 BRACKETING METHODS

5.2.3 Minimizing Function Evaluations

The bisection algorithm in Fig. 5.10 is just fi ne if you are performing a single root evaluation for a function that is easy to evaluate. However, there are many instances in engineering when this is not the case. For example, suppose that you develop a computer program that must locate a root numerous times. In such cases you could call the algorithm from Fig. 5.10 thousands and even millions of times in the course of a single run. Further, in its most general sense, a univariate function is merely an entity that re- turns a single value in return for a single value you send to it. Perceived in this sense, functions are not always simple formulas like the one-line equations solved in the pre- ceding examples in this chapter. For example, a function might consist of many lines of code that could take a signifi cant amount of execution time to evaluate. In some cases, the function might even represent an independent computer program. Because of both these factors, it is imperative that numerical algorithms minimize function evaluations. In this light, the algorithm from Fig. 5.10 is defi cient. In particular, notice that in making two function evaluations per iteration, it recalculates one of the functions that was determined on the previous iteration. Figure 5.11 provides a modifi ed algorithm that does not have this defi ciency. We have highlighted the lines that differ from Fig. 5.10. In this case, only the new function value at

FUNCTION Bisect(xl, xu, es, imax, xr, iter, ea) iter 5 0 fl 5 f(xl) DO xrold 5 xr xr 5 (xl 1 xu) / 2 fr 5 f(xr) iter 5 iter 1 1 IF xr ? 0 THEN ea 5 ABS((xr 2 xrold) / xr) * 100 END IF test 5 fl * fr IF test , 0 THEN xu 5 xr ELSE IF test . 0 THEN xl 5 xr fl 5 fr ELSE ea 5 0 END IF IF ea , es OR iter $ imax EXIT END DO Bisect 5 xr END Bisect

FIGURE 5.11 Pseudocode for bisection sub- program which minimizes function evaluations.

cha9792x_ch05_123-144.indd Page 134 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 134 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.3 THE FALSE-POSITION METHOD 135

the root estimate is calculated. Previously calculated values are saved and merely reassigned as the bracket shrinks. Thus, n 1 1 function evaluations are performed, rather than 2n.

5.3 THE FALSE-POSITION METHOD

Although bisection is a perfectly valid technique for determining roots, its “brute-force” approach is relatively ineffi cient. False position is an alternative based on a graphical insight. A shortcoming of the bisection method is that, in dividing the interval from xl to xu into equal halves, no account is taken of the magnitudes of f(xl) and f(xu). For example, if f(xl) is much closer to zero than f(xu), it is likely that the root is closer to xl than to xu (Fig. 5.12). An alternative method that exploits this graphical insight is to join f(xl) and f(xu) by a straight line. The intersection of this line with the x axis represents an improved estimate of the root. The fact that the replacement of the curve by a straight line gives a “false position” of the root is the origin of the name, method of false position, or in Latin, regula falsi. It is also called the linear interpolation method. Using similar triangles (Fig. 5.12), the intersection of the straight line with the x axis can be estimated as

f(xl) xr 2 xl

5 f(xu)

xr 2 xu (5.6)

which can be solved for (see Box 5.1 for details).

xr 5 xu 2 f(xu)(xl 2 xu)

f(xl) 2 f(xu) (5.7)

FIGURE 5.12 A graphical depiction of the method of false position. Similar triangles used to derive the for- mula for the method are shaded.

x

f (x)

f (xl)

f (xu)

xu

xl

xr

cha9792x_ch05_123-144.indd Page 135 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 135 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

136 BRACKETING METHODS

This is the false-position formula. The value of xr computed with Eq. (5.7) then replaces whichever of the two initial guesses, xl or xu, yields a function value with the same sign as f(xr). In this way, the values of xl and xu always bracket the true root. The process is repeated until the root is estimated adequately. The algorithm is identical to the one for bisection (Fig. 5.5) with the exception that Eq. (5.7) is used for step 2. In addition, the same stopping criterion [Eq. (5.2)] is used to terminate the computation.

EXAMPLE 5.5 False Position

Problem Statement. Use the false-position method to determine the root of the same equation investigated in Example 5.1 [Eq. (E5.1.1)].

Solution. As in Example 5.3, initiate the computation with guesses of xl 5 12 and xu 5 16.

First iteration:

xl 5 12    f (xl) 5 6.1139

xu 5 16   f (xu) 5 22.2303

xr 5 16 2 22.2303(12 2 16)

6.1139 2 (22.2303) 5 14.309

which has a true relative error of 0.88 percent.

Second iteration:

f(xl) f(xr) 5 21.5376

Box 5.1 Derivation of the Method of False Position

Cross-multiply Eq. (5.6) to yield

f (xl) (xr 2 xu) 5 f (xu) (xr 2 xl)

Collect terms and rearrange:

xr [ f(xl) 2 f (xu) ] 5 xu f (xl) 2 xl f (xu)

Divide by f(xl) 2 f(xu):

xr 5 xu f (xl) 2 xl f (xu)

f (xl) 2 f (xu) (B5.1.1)

This is one form of the method of false position. Note that it al- lows the computation of the root xr as a function of the lower and upper guesses xl and xu. It can be put in an alternative form by expanding it:

xr 5 xu f (xl)

f (xl) 2 f (xu) 2

xl f (xu)

f (xl) 2 f (xu)

then adding and subtracting xu on the right-hand side:

xr 5 xu 1 xu f (xl)

f (xl) 2 f (xu) 2 xu 2

xl f (xu)

f (xl) 2 f (xu)

Collecting terms yields

xr 5 xu 1 xu f (xu)

f (xl) 2 f (xu) 2

xl f (xu)

f (xl) 2 f(xu) or

xr 5 xu 2 f (xu) (xl 2 xu)

f (xl) 2 f (xu)

which is the same as Eq. (5.7). We use this form because it involves one less function evaluation and one less multiplication than Eq. (B5.1.1). In addition, it is directly comparable with the secant method, which will be discussed in Chap. 6.

cha9792x_ch05_123-144.indd Page 136 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 136 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.3 THE FALSE-POSITION METHOD 137

Therefore, the root lies in the fi rst subinterval, and xr becomes the upper limit for the next iteration, xu 5 14.9113:

xl 5 12       f(xl) 5 6.1139

xu 5 14.9309   f(xu) 5 20.2515

xr 5 14.9309 2 20.2515(12 2 14.9309)

6.1139 2 (20.2515) 5 14.8151

which has true and approximate relative errors of 0.09 and 0.78 percent. Additional iterations can be performed to refi ne the estimate of the roots.

FIGURE 5.13 Comparison of the relative errors of the bisection and the false-position methods.

63 Iterations

T ru

e p

er ce

n t

re la

ti ve

e rr

o r

0

10– 2

10– 3

Bisection

False position

10

1

10– 1

10– 4

A feeling for the relative effi ciency of the bisection and false-position methods can be appreciated by referring to Fig. 5.13, where we have plotted the true percent relative errors for Examples 5.4 and 5.5. Note how the error for false position decreases much faster than for bisection because of the more effi cient scheme for root location in the false-position method. Recall in the bisection method that the interval between xl and xu grew smaller during the course of a computation. The interval, as defi ned by ¢xy2 5 Z xu 2 xl Z y2 for the fi rst iteration, therefore provided a measure of the error for this approach. This is not the case

cha9792x_ch05_123-144.indd Page 137 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 137 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

138 BRACKETING METHODS

for the method of false position because one of the initial guesses may stay fi xed through- out the computation as the other guess converges on the root. For instance, in Example 5.5 the lower guess xl remained at 12 while xu converged on the root. For such cases, the interval does not shrink but rather approaches a constant value. Example 5.5 suggests that Eq. (5.2) represents a very conservative error criterion. In fact, Eq. (5.2) actually constitutes an approximation of the discrepancy of the previous iteration. This is because for a case such as Example 5.5, where the method is converg- ing quickly (for example, the error is being reduced nearly an order of magnitude per iteration), the root for the present iteration xnewr is a much better estimate of the true value than the result of the previous iteration xoldr . Thus, the quantity in the numerator of Eq. (5.2) actually represents the discrepancy of the previous iteration. Consequently, we are assured that satisfaction of Eq. (5.2) ensures that the root will be known with greater accuracy than the prescribed tolerance. However, as described in the next section, there are cases where false position converges slowly. For these cases, Eq. (5.2) becomes unreliable, and an alternative stopping criterion must be developed.

5.3.1 Pitfalls of the False-Position Method

Although the false-position method would seem to always be the bracketing method of preference, there are cases where it performs poorly. In fact, as in the following example, there are certain cases where bisection yields superior results.

EXAMPLE 5.6 A Case Where Bisection Is Preferable to False Position

Problem Statement. Use bisection and false position to locate the root of

f(x) 5 x10 2 1

between x 5 0 and 1.3.

Solution. Using bisection, the results can be summarized as

Iteration xl xu xr �a (%) �t (%)

1 0 1.3 0.65 100.0 35 2 0.65 1.3 0.975 33.3 2.5 3 0.975 1.3 1.1375 14.3 13.8 4 0.975 1.1375 1.05625 7.7 5.6 5 0.975 1.05625 1.015625 4.0 1.6

Thus, after fi ve iterations, the true error is reduced to less than 2 percent. For false position, a very different outcome is obtained:

Iteration xl xu xr �a (%) �t (%)

1 0 1.3 0.09430 90.6 2 0.09430 1.3 0.18176 48.1 81.8 3 0.18176 1.3 0.26287 30.9 73.7 4 0.26287 1.3 0.33811 22.3 66.2 5 0.33811 1.3 0.40788 17.1 59.2

cha9792x_ch05_123-144.indd Page 138 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 138 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.3 THE FALSE-POSITION METHOD 139

After fi ve iterations, the true error has only been reduced to about 59 percent. In addition, note that ea , et. Thus, the approximate error is misleading. Insight into these results can be gained by examining a plot of the function. As in Fig. 5.14, the curve violates the premise upon which false position was based—that is, if f(xl) is much closer to zero than f(xu), then the root is closer to xl than to xu (recall Fig. 5.12). Because of the shape of the present function, the opposite is true.

FIGURE 5.14 Plot of f (x) 5 x10 2 1, illustrating slow convergence of the false-position method.

1.0

10

5

0

f (x)

x

The forgoing example illustrates that blanket generalizations regarding root-location methods are usually not possible. Although a method such as false position is often supe- rior to bisection, there are invariably cases that violate this general conclusion. Therefore, in addition to using Eq. (5.2), the results should always be checked by substituting the root estimate into the original equation and determining whether the result is close to zero. Such a check should be incorporated into all computer programs for root location. The example also illustrates a major weakness of the false-position method: its one- sidedness. That is, as iterations are proceeding, one of the bracketing points will tend to

cha9792x_ch05_123-144.indd Page 139 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 139 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

140 BRACKETING METHODS

stay fi xed. This can lead to poor convergence, particularly for functions with signifi cant curvature. The following section provides a remedy.

5.3.2 Modifi ed False Position

One way to mitigate the “one-sided” nature of false position is to have the algorithm detect when one of the bounds is stuck. If this occurs, the function value at the stagnant bound can be divided in half. This is called the modifi ed false-position method. The algorithm in Fig. 5.15 implements this strategy. Notice how counters are used to determine when one of the bounds stays fi xed for two iterations. If this occurs, the function value at this stagnant bound is halved. The effectiveness of this algorithm can be demonstrated by applying it to Example 5.6. If a stopping criterion of 0.01% is used, the bisection and standard false-position

FUNCTION ModFalsePos(xl, xu, es, imax, xr, iter, ea) iter 5 0 fl 5 f(xl) fu 5 f(xu) DO xrold 5 xr xr 5 xu 2 fu * (xl 2 xu) / (fl 2 fu) fr 5 f(xr) iter 5 iter 1 1 IF xr , . 0 THEN ea 5 Abs((xr 2 xrold) / xr) * 100 END IF test 5 fl * fr IF test , 0 THEN xu 5 xr fu 5 f(xu) iu 5 0 il 5 il 1 1 If il $ 2 THEN fl 5 fl / 2 ELSE IF test . 0 THEN xl 5 xr fl 5 f(xl) il 5 0 iu 5 iu 1 1 IF iu $ 2 THEN fu 5 fu / 2 ELSE ea 5 0 END IF IF ea , es OR iter $ imax THEN EXIT END DO ModFalsePos 5 xr End MODFALSEPOS

FIGURE 5.15 Pseudocode for the modifi ed false-position method.

cha9792x_ch05_123-144.indd Page 140 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 140 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

5.4 INCREMENTAL SEARCHES AND DETERMINING INITIAL GUESSES 141

methods would converge in 14 and 39 iterations, respectively. In contrast, the modifi ed false-position method would converge in 12 iterations. Thus, for this example, it is somewhat more effi cient than bisection and is vastly superior to the unmodifi ed false- position method.

5.4 INCREMENTAL SEARCHES AND DETERMINING INITIAL GUESSES

Besides checking an individual answer, you must determine whether all possible roots have been located. As mentioned previously, a plot of the function is usually very useful in guiding you in this task. Another option is to incorporate an incremental search at the beginning of the computer program. This consists of starting at one end of the region of interest and then making function evaluations at small increments across the region. When the function changes sign, it is assumed that a root falls within the increment. The x values at the beginning and the end of the increment can then serve as the initial guesses for one of the bracketing techniques described in this chapter. A potential problem with an incremental search is the choice of the increment length. If the length is too small, the search can be very time consuming. On the other hand, if the length is too great, there is a possibility that closely spaced roots might be missed (Fig. 5.16). The problem is compounded by the possible existence of multiple roots. A partial remedy for such cases is to compute the fi rst derivative of the function f '(x) at the beginning and the end of each interval. If the derivative changes sign, it suggests that a minimum or maximum may have occurred and that the interval should be examined more closely for the existence of a possible root. Although such modifi cations or the employment of a very fi ne increment can allevi- ate the problem, it should be clear that brute-force methods such as incremental search are not foolproof. You would be wise to supplement such automatic techniques with any other information that provides insight into the location of the roots. Such information can be found in plotting and in understanding the physical problem from which the equation originated.

FIGURE 5.16 Cases where roots could be missed because the increment length of the search procedure is too large. Note that the last root on the right is multiple and would be missed regardless of increment length.

x6x0 x1 x2 x3 x4 x5

f (x)

x

cha9792x_ch05_123-144.indd Page 141 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 141 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

142 BRACKETING METHODS

PROBLEMS

5.1 Determine the real roots of f (x) 5 20.5x2 1 2.5x 1 4.5: (a) Graphically. (b) Using the quadratic formula. (c) Using three iterations of the bisection method to determine the

highest root. Employ initial guesses of xl 5 5 and xu 5 10. Compute the estimated error ea and the true error et after each iteration.

5.2 Determine the real root of f (x) 5 5x3 2 5x2 1 6x 2 2: (a) Graphically. (b) Using bisection to locate the root. Employ initial guesses of

xl 5 0 and xu 5 1 and iterate until the estimated error ea falls below a level of es 5 10%.

5.3 Determine the real root of f (x) 5 225 1 82x 2 90x2 1 44x3 2 8x4 1 0.7x5: (a) Graphically. (b) Using bisection to determine the root to es 5 10%. Employ

initial guesses of xl 5 0.5 and xu 5 1.0. (c) Perform the same computation as in (b) but use the false-

position method and es 5 0.2%. 5.4 (a) Determine the roots of f (x) 5 212 2 21x 1 18x2 2 2.75x3 graphically. In addition, determine the fi rst root of the function with (b) bisection, and (c) false position. For (b) and (c) use initial guesses of xl 5 21 and xu 5 0, and a stopping criterion of 1%. 5.5 Locate the fi rst nontrivial root of sin x 5 x2 where x is in radi- ans. Use a graphical technique and bisection with the initial interval from 0.5 to 1. Perform the computation until ea is less than es 5 2%. Also perform an error check by substituting your fi nal answer into the original equation. 5.6 Determine the positive real root of ln (x2) 5 0.7 (a) graphi- cally, (b) using three iterations of the bisection method, with initial guesses of xl 5 0.5 and xu 5 2, and (c) using three iterations of the false-position method, with the same initial guesses as in (b). 5.7 Determine the real root of f (x) 5 (0.8 2 0.3x)yx: (a) Analytically. (b) Graphically. (c) Using three iterations of the false-position method and initial

guesses of 1 and 3. Compute the approximate error ea and the true error et after each iteration. Is there a problem with the result?

5.8 Find the positive square root of 18 using the false-position method to within es 5 0.5%. Employ initial guesses of xl 5 4 and xu 5 5. 5.9 Find the smallest positive root of the function (x is in radians) x2Zcos 1x Z 5 5 using the false-position method. To locate the re- gion in which the root lies, fi rst plot this function for values of x between 0 and 5. Perform the computation until ea falls below es 5 1%. Check your fi nal answer by substituting it into the orig- inal function.

5.10 Find the positive real root of f (x) 5 x4 2 8x3 2 35x2 1 450x 2 1001 using the false-position method. Use initial guesses of xl 5 4.5 and xu 5 6 and perform fi ve iterations. Compute both the true and approximate errors based on the fact that the root is 5.60979. Use a plot to explain your results and perform the compu- tation to within es 5 1.0%. 5.11 Determine the real root of x3.5 5 80: (a) analytically and (b) with the false-position method to within es 5 2.5%. Use initial guesses of 2.0 and 5.0. 5.12 Given

f (x) 5 22x6 2 1.5x4 1 10x 1 2

Use bisection to determine the maximum of this function. Employ initial guesses of xl 5 0 and xu 5 1, and perform iterations until the approximate relative error falls below 5%. 5.13 The velocity y of a falling parachutist is given by

y 5 gm

c (1 2 e2 (cym)t)

where g 5 9.81 mys2. For a parachutist with a drag coeffi cient c 5 15 kg/s, compute the mass m so that the velocity is y 5 36 m/s at t 5 10 s. Use the false-position method to determine m to a level of es 5 0.1%. 5.14 Use bisection to determine the drag coeffi cient needed so that an 82-kg parachutist has a velocity of 36 m/s after 4 s of free fall. Note: The acceleration of gravity is 9.81 m/s2. Start with initial guesses of xl 5 3 and xu 5 5 and iterate until the approximate relative error falls below 2%. Also perform an error check by sub- stituting your fi nal answer into the original equation. 5.15 As depicted in Fig. P5.15, the velocity of water, y (m/s), discharged from a cylindrical tank through a long pipe can be computed as

y 5 12gH tanh a12gH 2L

tb

H L

v

FIGURE P5.15

cha9792x_ch05_123-144.indd Page 142 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 142 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 143

your answer. Determine the approximate relative error after each iteration. Employ initial guesses of 0 and R. 5.18 The saturation concentration of dissolved oxygen in freshwa- ter can be calculated with the equation (APHA, 1992)

ln osf 5 2139.34411 1 1.575701 3 105

Ta 2

6.642308 3 107

T 2a

1 1.243800 3 1010

T 3a 2

8.621949 3 1011

T 4a

where osf 5 the saturation concentration of dissolved oxygen in freshwater at 1 atm (mg/L) and Ta 5 absolute temperature (K). Remember that Ta 5 T 1 273.15, where T 5 temperature (°C). According to this equation, saturation decreases with increasing temperature. For typical natural waters in temperate climates, the equation can be used to determine that oxygen concentration ranges from 14.621 mg/L at 0°C to 6.413 mg/L at 40°C. Given a value of oxygen concentration, this formula and the bisection method can be used to solve for temperature in °C. (a) If the initial guesses are set as 0 and 408C, how many bisection

iterations would be required to determine temperature to an absolute error of 0.058C?

(b) Develop and test a bisection program to determine T as a func- tion of a given oxygen concentration to a prespecifi ed absolute error as in (a). Given initial guesses of 0 and 408C, test your program for an absolute error 5 0.058C and the following cases: osf 5 8, 10, and 12 mg/L. Check your results.

5.19 According to Archimedes principle, the buoyancy force is equal to the weight of fl uid displaced by the submerged portion of an object. For the sphere depicted in Fig. P5.19, use bisection to deter- mine the height h of the portion that is above water. Employ the follow- ing values for your computation: r 5 1 m, �s 5 density of sphere 5 200 kg/m3, and �w 5 density of water 5 1000 kg/m

3. Note that the volume of the above-water portion of the sphere can be computed with

V 5 ph2

3 (3r 2 h)

h

r

FIGURE P5.19

where g 5 9.81 m/s2, H 5 initial head (m), L 5 pipe length (m), and t 5 elapsed time (s). Determine the head needed to achieve y  5 5 m/s in 2.5 s for a 4-m-long pipe (a) graphically, (b) by bisection, and (c) with false position. Employ initial guesses of xl 5 0 and xu 5 2 m with a stopping criterion of es 5 1%. Check you results. 5.16 Water is fl owing in a trapezoidal channel at a rate of Q 5 20 m3/s. The critical depth y for such a channel must satisfy the equation

0 5 1 2 Q2

gA3c B

where g 5 9.81 m/s2, Ac 5 the cross-sectional area (m 2), and B 5

the width of the channel at the surface (m). For this case, the width and the cross-sectional area can be related to depth y by

B 5 3 1 y  and  Ac 5 3y 1 y2

2

Solve for the critical depth using (a) the graphical method, (b) bisec- tion, and (c) false position. For (b) and (c) use initial guesses of xl 5 0.5 and xu 5 2.5, and iterate until the approximate error falls below 1% or the number of iterations exceeds 10. Discuss your results. 5.17 You are designing a spherical tank (Fig. P5.17) to hold water for a small village in a developing country. The volume of liquid it can hold can be computed as

V 5 ph2 [3R 2 h]

3

where V 5 volume (m3), h 5 depth of water in tank (m), and R 5 the tank radius (m).

hV

R

FIGURE P5.17

If R 5 3 m, to what depth must the tank be fi lled so that it holds 30 m3? Use three iterations of the false-position method to determine

cha9792x_ch05_123-144.indd Page 143 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 143 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

144 BRACKETING METHODS

(c) Add an answer check that substitutes the root estimate into the original function to verify whether the fi nal result is close to zero.

(d) Test the subprogram by duplicating the computations from Examples 5.3 and 5.4.

5.22 Develop a subprogram for the bisection method that mini- mizes function evaluations based on the pseudocode from Fig. 5.11. Determine the number of function evaluations (n) per total itera- tions. Test the program by duplicating Example 5.6. 5.23 Develop a user-friendly program for the false-position method. The structure of your program should be similar to the bisection algorithm outlined in Fig. 5.10. Test the program by duplicating Example 5.5. 5.24 Develop a subprogram for the false-position method that min- imizes function evaluations in a fashion similar to Fig. 5.11. Deter- mine the number of function evaluations (n) per total iterations. Test the program by duplicating Example 5.6. 5.25 Develop a user-friendly subprogram for the modifi ed false- position method based on Fig. 5.15. Test the program by deter- mining the root of the function described in Example 5.6. Perform a number of runs until the true percent relative error falls below 0.01%. Plot the true and approximate percent relative errors versus number of iterations on semilog paper. Interpret your results. 5.26 Develop a function for bisection in a similar fashion to Fig. 5.10. However, rather than using the maximum iterations and Eq. (5.2), employ Eq. (5.5) as your stopping criterion. Make sure to round the result of Eq. (5.5) up to the next highest integer. Test your function by solving Example 5.3 using Ea,d 5 0.0001.

5.20 Perform the same computation as in Prob. 5.19, but for the frustrum of a cone, as depicted in Fig. P5.20. Employ the following values for your computation: r1 5 0.5 m, r2 5 1 m, h 5 1 m, �f 5 frustrum density 5 200 kg/m3, and �w 5 water density 5 1000 kg/m

3. Note that the volume of a frustrum is given by

V 5 ph

3 (r21 1 r22 1 r1r2)

h

h1

r2

r1

FIGURE P5.20

5.21 Integrate the algorithm outlined in Fig. 5.10 into a complete, user-friendly bisection subprogram. Among other things: (a) Place documentation statements throughout the subprogram to

identify what each section is intended to accomplish. (b) Label the input and output.

cha9792x_ch05_123-144.indd Page 144 15/10/13 7:11 PM f-479 cha9792x_ch05_123-144.indd Page 144 15/10/13 7:11 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6 C H A P T E R 6

145

Open Methods

For the bracketing methods in Chap. 5, the root is located within an interval prescribed by a lower and an upper bound. Repeated application of these methods always results in closer estimates of the true value of the root. Such methods are said to be convergent because they move closer to the truth as the computation progresses (Fig. 6.1a). In contrast, the open methods described in this chapter are based on formulas that require only a single starting value of x or two starting values that do not

FIGURE 6.1 Graphical depiction of the fundamental difference between the (a) bracketing and (b) and (c) open methods for root location. In (a), which is the bisection method, the root is constrained within the interval prescribed by xl and xu. In contrast, for the open method depicted in (b) and (c), a formula is used to project from xi to xi11 in an iterative fashion. Thus, the method can either (b) diverge or (c) converge rapidly, depending on the value of the initial guess.

f (x)

x

(a)

xl xu

xl xu

f (x)

x

(b)

xi

xi + 1

f (x)

x

(c)

xi

xi + 1

xl xu

xl xu

xl xu

cha9792x_ch06_145-175.indd Page 145 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 145 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

146 OPEN METHODS

necessarily bracket the root. As such, they sometimes diverge or move away from the true root as the computation progresses (Fig. 6.1b). However, when the open methods converge (Fig. 6.1c), they usually do so much more quickly than the brack- eting methods. We will begin our discussion of open techniques with a simple version that is useful for illustrating their general form and also for demonstrating the con- cept of convergence.

6.1 SIMPLE FIXED-POINT ITERATION

As mentioned above, open methods employ a formula to predict the root. Such a formula can be developed for simple fi xed-point iteration (or, as it is also called, one-point it- eration or successive substitution) by rearranging the function f(x) 5 0 so that x is on the left-hand side of the equation:

x 5 g(x) (6.1)

This transformation can be accomplished either by algebraic manipulation or by simply adding x to both sides of the original equation. For example,

x2 2 2x 1 3 5 0

can be simply manipulated to yield

x 5 x2 1 3

2

whereas sin x 5 0 could be put into the form of Eq. (6.1) by adding x to both sides to yield

x 5 sin x 1 x

The utility of Eq. (6.1) is that it provides a formula to predict a new value of x as a function of an old value of x. Thus, given an initial guess at the root xi, Eq. (6.1) can be used to compute a new estimate xi11 as expressed by the iterative formula

xi11 5 g(xi) (6.2)

As with other iterative formulas in this book, the approximate error for this equation can be determined using the error estimator [Eq. (3.5)]:

ea 5 ` xi11 2 xixi11 ` 100%

EXAMPLE 6.1 Simple Fixed-Point Iteration

Problem Statement. Use simple fi xed-point iteration to locate the root of f(x) 5 e2x 2 x.

Solution. The function can be separated directly and expressed in the form of Eq. (6.2) as

xi11 5 e 2xi

cha9792x_ch06_145-175.indd Page 146 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 146 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.1 SIMPLE FIXED-POINT ITERATION 147

Starting with an initial guess of x0 5 0, this iterative equation can be applied to compute

i xi Ea (%) Et (%)

0 0 100.0 1 1.000000 100.0 76.3 2 0.367879 171.8 35.1 3 0.692201 46.9 22.1 4 0.500473 38.3 11.8 5 0.606244 17.4 6.89 6 0.545396 11.2 3.83 7 0.579612 5.90 2.20 8 0.560115 3.48 1.24 9 0.571143 1.93 0.705 10 0.564879 1.11 0.399

Thus, each iteration brings the estimate closer to the true value of the root: 0.56714329.

6.1.1 Convergence

Notice that the true percent relative error for each iteration of Example 6.1 is roughly proportional (by a factor of about 0.5 to 0.6) to the error from the previous iteration. This property, called linear convergence, is characteristic of fi xed-point iteration. Aside from the “rate” of convergence, we must comment at this point about the “possibility” of convergence. The concepts of convergence and divergence can be de- picted graphically. Recall that in Sec. 5.1, we graphed a function to visualize its structure and behavior (Example 5.1). Such an approach is employed in Fig. 6.2a for the function f(x) 5 e2x 2 x. An alternative graphical approach is to separate the equation into two component parts, as in

f1(x) 5 f2(x)

Then the two equations

y1 5 f1(x) (6.3)

and

y2 5 f2(x) (6.4)

can be plotted separately (Fig. 6.2b). The x values corresponding to the intersections of these functions represent the roots of f(x) 5 0.

EXAMPLE 6.2 The Two-Curve Graphical Method

Problem Statement. Separate the equation e2x 2 x 5 0 into two parts and determine its root graphically.

cha9792x_ch06_145-175.indd Page 147 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 147 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

148 OPEN METHODS

These points are plotted in Fig. 6.2b. The intersection of the two curves indicates a root estimate of approximately x 5 0.57, which corresponds to the point where the single curve in Fig. 6.2a crosses the x axis.

Solution. Reformulate the equation as y1 5 x and y2 5 e2x. The following values can be computed:

x y1 y2

0.0 0.0 1.000 0.2 0.2 0.819 0.4 0.4 0.670 0.6 0.6 0.549 0.8 0.8 0.449 1.0 1.0 0.368

FIGURE 6.2 Two alternative graphical methods for determining the root of f(x) 5 e2x 2 x. (a) Root at the point where it crosses the x axis; (b) root at the intersec- tion of the component functions.

f (x)

f (x)

x

x

Root

Root

f (x) = e– x – x

f 1(x) = x

f 2(x) = e – x

(a)

(b)

cha9792x_ch06_145-175.indd Page 148 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 148 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.1 SIMPLE FIXED-POINT ITERATION 149

The two-curve method can now be used to illustrate the convergence and divergence of fi xed-point iteration. First, Eq. (6.1) can be reexpressed as a pair of equations y1 5 x and y2 5 g(x). These two equations can then be plotted separately. As was the case with Eqs. (6.3) and (6.4), the roots of f(x) 5 0 correspond to the abscissa value at the inter- section of the two curves. The function y1 5 x and four different shapes for y2 5 g(x) are plotted in Fig. 6.3. For the fi rst case (Fig. 6.3a), the initial guess of x0 is used to determine the corre- sponding point on the y2 curve [x0, g(x0)]. The point (x1, x1) is located by moving left horizontally to the y1 curve. These movements are equivalent to the fi rst iteration in the fi xed-point method:

x1 5 g(x0)

Thus, in both the equation and in the plot, a starting value of x0 is used to obtain an estimate of x1. The next iteration consists of moving to [x1, g(x1)] and then to (x2, x2). This iteration is equivalent to the equation

x2 5 g(x1)

FIGURE 6.3 Iteration cobwebs depicting convergence (a and b) and divergence (c and d) of simple fi xed-point iteration. Graphs (a) and (c) are called monotone patterns, whereas (b) and (d) are called oscillating or spiral patterns. Note that convergence occurs when |g9(x)| , 1.

xx1

y1 = x

y2 = g(x)

x2 x0

y

(a) x

y1 = x

y2 = g(x)

x0

y

(b)

x

y1 = x

y2 = g(x)

x0

y

(c) x

y1 = x

y2 = g(x)

x0

y

(d)

cha9792x_ch06_145-175.indd Page 149 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 149 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

150 OPEN METHODS

The solution in Fig. 6.3a is convergent because the estimates of x move closer to the root with each iteration. The same is true for Fig. 6.3b. However, this is not the case for Fig. 6.3c and d, where the iterations diverge from the root. Notice that convergence seems to occur only when the absolute value of the slope of y2 5 g(x) is less than the slope of y1 5 x, that is, when ug9(x)u , 1. Box 6.1 provides a theoretical deriva- tion of this result.

6.1.2 Algorithm for Fixed-Point Iteration

The computer algorithm for fi xed-point iteration is extremely simple. It consists of a loop to iteratively compute new estimates until the termination criterion has been met. Figure 6.4 presents pseudocode for the algorithm. Other open methods can be pro- grammed in a similar way, the major modifi cation being to change the iterative formula that is used to compute the new root estimate.

Box 6.1 Convergence of Fixed-Point Iteration

From studying Fig. 6.3, it should be clear that fi xed-point iteration converges if, in the region of interest, ug9(x)u , 1. In other words, convergence occurs if the magnitude of the slope of g(x) is less than the slope of the line f(x) 5 x. This observation can be demonstrated theoretically. Recall that the iterative equation is

xi11 5 g(xi)

Suppose that the true solution is

xr 5 g(xr)

Subtracting these equations yields

xr 2 xi11 5 g(xr) 2 g(xi) (B6.1.1)

The derivative mean-value theorem (recall Sec. 4.1.1) states that if a function g(x) and its fi rst derivative are continuous over an inter- val a # x # b, then there exists at least one value of x 5 j within the interval such that

g¿(j) 5 g(b) 2 g(a)

b 2 a (B6.1.2)

The right-hand side of this equation is the slope of the line joining g(a) and g(b). Thus, the mean-value theorem states that there is at least one point between a and b that has a slope, designated by g9(j), which is parallel to the line joining g(a) and g(b) (recall Fig. 4.3).

Now, if we let a 5 xi and b 5 xr, the right-hand side of Eq. (B6.1.1) can be expressed as

g(xr) 2 g(xi) 5 (xr 2 xi)g¿(j)

where j is somewhere between xi and xr. This result can then be substituted into Eq. (B6.1.1) to yield

xr 2 xi11 5 (xr 2 xi)g¿(j) (B6.1.3)

If the true error for iteration i is defi ned as

Et,i 5 xr 2 xi

then Eq. (B6.1.3) becomes

Et,i11 5 g¿(j)Et,i

Consequently, if ug9(x)u , 1, the errors decrease with each iteration. For ug9(x)u . 1, the errors grow. Notice also that if the derivative is positive, the errors will be positive, and hence, the iterative solution will be monotonic (Fig. 6.3a and c). If the derivative is negative, the errors will oscillate (Fig. 6.3b and d). An offshoot of the analysis is that it also demonstrates that when the method converges, the error is roughly proportional to and less than the error of the previous step. For this reason, simple fi xed- point iteration is said to be linearly convergent.

cha9792x_ch06_145-175.indd Page 150 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 150 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.2 THE NEWTON-RAPHSON METHOD 151

6.2 THE NEWTON-RAPHSON METHOD

Perhaps the most widely used of all root-locating formulas is the Newton-Raphson equa- tion (Fig. 6.5). If the initial guess at the root is xi, a tangent can be extended from the point [xi, f(xi)]. The point where this tangent crosses the x axis usually represents an improved estimate of the root.

FUNCTION Fixpt(x0, es, imax, iter, ea) xr 5 x0 iter 5 0 DO xrold 5 xr xr 5 g(xrold) iter 5 iter 1 1 IF xr ? O THEN

ea 5 ` xr 2 xrold xr

` ? 100 END IF IF ea , es OR iter $ imax EXIT END DO Fixpt 5 xr END Fixpt

FIGURE 6.4 Pseudocode for fi xed-point iteration. Note that other open methods can be cast in this general format.

f (x)

f (xi)

f (xi) – 0

Slope = f ' (xi)

0 xxi+1 xi

xi – xi+1

FIGURE 6.5 Graphical depiction of the Newton-Raphson method. A tangent to the function of xi [that is, f9(xi)] is extrapolated down to the x axis to provide an estimate of the root at xi11.

cha9792x_ch06_145-175.indd Page 151 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 151 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

152 OPEN METHODS

The Newton-Raphson method can be derived on the basis of this geometrical inter- pretation (an alternative method based on the Taylor series is described in Box 6.2). As in Fig. 6.5, the fi rst derivative at x is equivalent to the slope:

f ¿(xi) 5 f (xi) 2 0 xi 2 xi11

(6.5)

which can be rearranged to yield

xi11 5 xi 2 f(xi)

f ¿(xi) (6.6)

which is called the Newton-Raphson formula.

EXAMPLE 6.3 Newton-Raphson Method

Problem Statement. Use the Newton-Raphson method to estimate the root of f(x) 5 e2x 2 x, employing an initial guess of x0 5 0.

Solution. The fi rst derivative of the function can be evaluated as

f ¿(x) 5 2e2x 2 1

which can be substituted along with the original function into Eq. (6.6) to give

xi11 5 xi 2 e2xi 2 xi

2e2xi 2 1

Starting with an initial guess of x0 5 0, this iterative equation can be applied to compute

i xi Et (%)

0 0 100 1 0.500000000 11.8 2 0.566311003 0.147 3 0.567143165 0.0000220 4 0.567143290 , 1028

Thus, the approach rapidly converges on the true root. Notice that the true percent relative error at each iteration decreases much faster than it does in simple fi xed-point iteration (compare with Example 6.1).

6.2.1 Termination Criteria and Error Estimates

As with other root-location methods, Eq. (3.5) can be used as a termination criterion. In addition, however, the Taylor series derivation of the method (Box 6.2) provides theo- retical insight regarding the rate of convergence as expressed by Ei11 5 O(E

2 i ). Thus the

error should be roughly proportional to the square of the previous error. In other words,

cha9792x_ch06_145-175.indd Page 152 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 152 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.2 THE NEWTON-RAPHSON METHOD 153

the number of signifi cant fi gures of accuracy approximately doubles with each iteration. This behavior is examined in the following example.

EXAMPLE 6.4 Error Analysis of Newton-Raphson Method

Problem Statement. As derived in Box 6.2, the Newton-Raphson method is quadrati- cally convergent. That is, the error is roughly proportional to the square of the previous error, as in

Et,i11 > 2f –(xr) 2f ¿(xr)

E2t,i (E6.4.1)

Examine this formula and see if it applies to the results of Example 6.3.

Solution. The fi rst derivative of f(x) 5 e2x 2 x is

f ¿(x) 5 2e2x 2 1

Box 6.2 Derivation and Error Analysis of the Newton-Raphson Method

Aside from the geometric derivation [Eqs. (6.5) and (6.6)], the Newton-Raphson method may also be developed from the Taylor series expansion. This alternative derivation is useful in that it also provides insight into the rate of convergence of the method. Recall from Chap. 4 that the Taylor series expansion can be represented as

f (xi11) 5 f (xi) 1 f ¿(xi) (xi11 2 xi)

1 f –(j)

2! (xi11 2 xi)

2 (B6.2.1)

where j lies somewhere in the interval from xi to xi11. An approxi- mate version is obtainable by truncating the series after the fi rst derivative term:

f (xi11) > f (xi) 1 f ¿(xi) (xi11 2 xi)

At the intersection with the x axis, f(xi11) would be equal to zero, or

0 5 f (xi) 1 f ¿(xi) (xi11 2 xi) (B6.2.2)

which can be solved for

xi11 5 xi 2 f (xi)

f ¿(xi)

which is identical to Eq. (6.6). Thus, we have derived the Newton- Raphson formula using a Taylor series. Aside from the derivation, the Taylor series can also be used to estimate the error of the formula. This can be done by realizing that if the complete Taylor series were employed, an exact result would

be obtained. For this situation xi11 5 xr, where x is the true value of the root. Substituting this value along with f(xr) 5 0 into Eq. (B6.2.1) yields

0 5 f (xi) 1 f ¿(xi) (xr 2 xi) 1 f –(j)

2! (xr 2 xi)

2 (B6.2.3)

Equation (B6.2.2) can be subtracted from Eq. (B6.2.3) to give

0 5 f ¿(xi) (xr 2 xi11) 1 f –(j)

2! (xr 2 xi)

2 (B6.2.4)

Now, realize that the error is equal to the discrepancy between xi11 and the true value xr, as in

Et,i11 5 xr 2 xi11

and Eq. (B6.2.4) can be expressed as

0 5 f ¿(xi)Et,i11 1 f –(j)

2! E2t,i (B6.2.5)

If we assume convergence, both xi and j should eventually be ap- proximated by the root xr, and Eq. (B6.2.5) can be rearranged to yield

Et,i11 5 2f –(xr) 2 f ¿(xr)

E2t,i (B6.2.6)

According to Eq. (B6.2.6), the error is roughly proportional to the square of the previous error. This means that the number of correct decimal places approximately doubles with each iteration. Such behavior is referred to as quadratic convergence. Example 6.4 manifests this property.

cha9792x_ch06_145-175.indd Page 153 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 153 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

154 OPEN METHODS

which can be evaluated at xr 5 0.56714329 as f 9(0.56714329) 5 21.56714329. The second derivative is

f –(x) 5 e2x

which can be evaluated as f 0(0.56714329) 5 0.56714329. These results can be substituted into Eq. (E6.4.1) to yield

Et,i11 > 2 0.56714329

2(21.56714329) E2t,i 5 0.18095E

2 t,i

From Example 6.3, the initial error was Et,0 5 0.56714329, which can be substituted into the error equation to predict

Et,1 > 0.18095(0.56714329) 2 5 0.0582

which is close to the true error of 0.06714329. For the next iteration,

Et,2 > 0.18095(0.06714329) 2 5 0.0008158

which also compares favorably with the true error of 0.0008323. For the third iteration,

Et,3 > 0.18095(0.0008323) 2 5 0.000000125

which is the error obtained in Example 6.3. The error estimate improves in this manner because, as we come closer to the root, x and j are better approximated by xr [recall our assumption in going from Eq. (B6.2.5) to Eq. (B6.2.6) in Box 6.2]. Finally,

Et,4 > 0.18095(0.000000125) 2 5 2.83 3 10215

Thus, this example illustrates that the error of the Newton-Raphson method for this case is, in fact, roughly proportional (by a factor of 0.18095) to the square of the error of the previous iteration.

6.2.2 Pitfalls of the Newton-Raphson Method

Although the Newton-Raphson method is often very effi cient, there are situations where it performs poorly. A special case—multiple roots—will be addressed later in this chapter. However, even when dealing with simple roots, diffi culties can also arise, as in the fol- lowing example.

EXAMPLE 6.5 Example of a Slowly Converging Function with Newton-Raphson

Problem Statement. Determine the positive root of f(x) 5 x10 2 1 using the Newton- Raphson method and an initial guess of x 5 0.5.

Solution. The Newton-Raphson formula for this case is

xi11 5 xi 2 x10i 2 1

10x9i

which can be used to compute

cha9792x_ch06_145-175.indd Page 154 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 154 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.2 THE NEWTON-RAPHSON METHOD 155

Aside from slow convergence due to the nature of the function, other diffi culties can arise, as illustrated in Fig. 6.6. For example, Fig. 6.6a depicts the case where an infl ection point [that is, f 0(x) 5 0] occurs in the vicinity of a root. Notice that iterations beginning at x0 progressively diverge from the root. Figure 6.6b illustrates the tendency of the Newton-Raphson technique to oscillate around a local maximum or minimum. Such oscillations may persist, or as in Fig. 6.6b, a near-zero slope is reached, whereupon the solution is sent far from the area of interest. Figure 6.6c shows how an initial guess that is close to one root can jump to a location several roots away. This tendency to move away from the area of interest is because near- zero slopes are encountered. Obviously, a zero slope [ f 9(x) 5 0] is truly a disaster because it causes division by zero in the Newton-Raphson formula [Eq. (6.6)]. Graphically (see Fig 6.6d), it means that the solution shoots off horizontally and never hits the x axis. Thus, there is no general convergence criterion for Newton-Raphson. Its convergence depends on the nature of the function and on the accuracy of the initial guess. The only remedy is to have an initial guess that is “suffi ciently” close to the root. And for some functions, no guess will work! Good guesses are usually predicated on knowledge of the physical problem setting or on devices such as graphs that provide insight into the be- havior of the solution. The lack of a general convergence criterion also suggests that good computer software should be designed to recognize slow convergence or diver- gence. The next section addresses some of these issues.

6.2.3 Algorithm for Newton-Raphson

An algorithm for the Newton-Raphson method is readily obtained by substituting Eq. (6.6) for the predictive formula [Eq. (6.2)] in Fig. 6.4. Note, however, that the program must also be modifi ed to compute the fi rst derivative. This can be simply accomplished by the inclusion of a user-defi ned function.

Iteration x

0 0.5 1 51.65 2 46.485 3 41.8365 4 37.65285 5 33.887565 . . . ` 1.0000000

Thus, after the fi rst poor prediction, the technique is converging on the true root of 1, but at a very slow rate.

cha9792x_ch06_145-175.indd Page 155 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 155 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

156 OPEN METHODS

Additionally, in light of the foregoing discussion of potential problems of the Newton- Raphson method, the program would be improved by incorporating several additional features:

f (x)

x

x2x0x1

(a)

f (x)

xx2 x4x0 x1x3

(b)

f (x)

xx0

x1x2

(c)

f (x)

xx0 x1

(d )

FIGURE 6.6 Four cases where the Newton-Raphson method exhibits poor convergence.

cha9792x_ch06_145-175.indd Page 156 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 156 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.3 THE SECANT METHOD 157

1. A plotting routine should be included in the program. 2. At the end of the computation, the fi nal root estimate should always be substituted

into the original function to compute whether the result is close to zero. This check partially guards against those cases where slow or oscillating convergence may lead to a small value of ea while the solution is still far from a root.

3. The program should always include an upper limit on the number of iterations to guard against oscillating, slowly convergent, or divergent solutions that could persist interminably.

4. The program should alert the user and take account of the possibility that f 9(x) might equal zero at any time during the computation.

6.3 THE SECANT METHOD

A potential problem in implementing the Newton-Raphson method is the evaluation of the derivative. Although this is not inconvenient for polynomials and many other func- tions, there are certain functions whose derivatives may be extremely diffi cult or incon- venient to evaluate. For these cases, the derivative can be approximated by a backward fi nite divided difference, as in (Fig. 6.7)

f ¿(xi) > f (xi21) 2 f (xi)

xi21 2 xi

This approximation can be substituted into Eq. (6.6) to yield the following iterative equation:

xi11 5 xi 2 f (xi)(xi21 2 xi)

f (xi21) 2 f (xi) (6.7)

f (x )

f (x i )

f (x i – 1)

xx ix i – 1

FIGURE 6.7 Graphical depiction of the se- cant method. This technique is similar to the Newton-Raphson technique (Fig. 6.5) in the sense that an estimate of the root is predicted by extrapolating a tangent of the function to the x axis. However, the secant method uses a difference rather than a derivative to estimate the slope.

cha9792x_ch06_145-175.indd Page 157 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 157 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

158 OPEN METHODS

Equation (6.7) is the formula for the secant method. Notice that the approach requires two initial estimates of x. However, because f(x) is not required to change signs between the estimates, it is not classifi ed as a bracketing method.

EXAMPLE 6.6 The Secant Method

Problem Statement. Use the secant method to estimate the root of f(x) 5 e2x 2 x. Start with initial estimates of x21 5 0 and x0 5 1.0.

Solution. Recall that the true root is 0.56714329. . . .

First iteration:

x21 5 0  f (x21) 5 1.00000

x0 5 1  f(x0) 5 20.63212

x1 5 1 2 20.63212(0 2 1)

1 2 (20.63212) 5 0.61270  et 5 8.0%

Second iteration:

x0 5 1  f  (x0) 5 20.63212

x1 5 0.61270  f  (x1) 5 20.07081

(Note that both estimates are now on the same side of the root.)

x2 5 0.61270 2 20.07081(1 2 0.61270)

20.63212 2 (20.07081) 5 0.56384  et 5 0.58%

Third iteration:

x1 5 0.61270  f  (x1) 5 20.07081

x2 5 0.56384  f  (x2) 5 0.00518

x3 5 0.56384 2 0.00518(0.61270 2 0.56384)

20.07081 2 (20.00518) 5 0.56717  et 5 0.0048%

6.3.1 The Difference Between the Secant and False-Position Methods

Note the similarity between the secant method and the false-position method. For example, Eqs. (6.7) and (5.7) are identical on a term-by-term basis. Both use two initial estimates to compute an approximation of the slope of the function that is used to project to the x  axis for a new estimate of the root. However, a critical difference between the methods is how one of the initial values is replaced by the new estimate. Recall that in the false-position method the latest estimate of the root replaces whichever of the original values yielded a function value with the same sign as f(xr). Consequently, the two estimates always bracket the root. Therefore, for all practical purposes, the method always converges because the root is kept within the bracket. In contrast, the secant method replaces the values in strict sequence, with the new value xi11 replacing xi and xi replacing xi21. As a result, the two values can sometimes lie on the same side of the root. For certain cases, this can lead to divergence.

cha9792x_ch06_145-175.indd Page 158 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 158 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

EXAMPLE 6.7 Comparison of Convergence of the Secant and False-Position Techniques

Problem Statement. Use the false-position and secant methods to estimate the root of f(x) 5 ln x. Start the computation with values of xl 5 xi21 5 0.5 and xu 5 xi 5 5.0.

Solution. For the false-position method, the use of Eq. (5.7) and the bracketing criterion for replacing estimates results in the following iterations:

Iteration xl xu xr

1 0.5 5.0 1.8546 2 0.5 1.8546 1.2163 3 0.5 1.2163 1.0585

As can be seen (Fig. 6.8a and c), the estimates are converging on the true root which is equal to 1.

6.3 THE SECANT METHOD 159

FIGURE 6.8 Comparison of the false-position and the secant methods. The fi rst iterations (a) and (b) for both techniques are identical. However, for the second iterations (c) and (d), the points used differ. As a consequence, the secant method can diverge, as indicated in (d).

f (x) f (xu )

f (xl )

xxr

(a)

False position f (x) f (xi )

f (xi )

f (xi – 1)

xxr

(b)

Secant

f (x)

f (xl )

f (xu )

xxr

(c)

f (x) f (xi – 1)

x

xr

(d )

cha9792x_ch06_145-175.indd Page 159 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 159 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

160 OPEN METHODS

For the secant method, using Eq. (6.7) and the sequential criterion for replacing estimates results in

Iteration xi�1 xi xi�1

1 0.5 5.0 1.8546 2 5.0 1.8546 �0.10438

As in Fig. 6.8d, the approach is divergent.

Although the secant method may be divergent, when it converges it usually does so at a quicker rate than the false-position method. For instance, Fig. 6.9 demonstrates the superiority of the secant method in this regard. The inferiority of the false-position method is due to one end staying fi xed to maintain the bracketing of the root. This property, which is an advantage in that it prevents divergence, is a shortcoming with regard to the rate of convergence; it makes the fi nite-difference estimate a less-accurate approximation of the derivative.

20 Iterations

T ru

e p

er ce

n t

re la

ti ve

e rr

o r

10– 6

10– 5

10– 4

10– 3

10– 2

10– 1

1

10

False position

S ecant

N ew

ton-R ap

h so

n

Bisection

FIGURE 6.9 Comparison of the true percent relative errors et for the methods to determine the roots of f (x) 5 e2x 2 x.

cha9792x_ch06_145-175.indd Page 160 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 160 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.3 THE SECANT METHOD 161

6.3.2 Algorithm for the Secant Method

As with the other open methods, an algorithm for the secant method is obtained simply by modifying Fig. 6.4 so that two initial guesses are input and by using Eq. (6.7) to calculate the root. In addition, the options suggested in Sec. 6.2.3 for the Newton-Raphson method can also be applied to good advantage for the secant program.

6.3.3 Modifi ed Secant Method

Rather than using two arbitrary values to estimate the derivative, an alternative approach involves a fractional perturbation of the independent variable to estimate f9(x),

f ¿(xi) > f(xi 1 dxi) 2 f(xi)

dxi

where d 5 a small perturbation fraction. This approximation can be substituted into Eq. (6.6) to yield the following iterative equation:

xi11 5 xi 2 dxi f (xi)

f  (xi 1 dxi) 2 f  (xi) (6.8)

EXAMPLE 6.8 Modifi ed Secant Method

Problem Statement. Use the modifi ed secant method to estimate the root of f(x) 5 e2x 2 x. Use a value of 0.01 for d and start with x0 5 1.0. Recall that the true root is 0.56714329. . . .

Solution. First iteration:

x0 5 1   f(x0) 5 20.63212

x0 1 dx0 5 1.01  f(x0 1 dx0) 5 20.64578

x1 5 1 2 0.01(20.63212)

20.64578 2 (20.63212) 5 0.537263  Zet Z 5 5.3%

Second iteration:

x0 5 0.537263   f(x0) 5 0.047083

x0 1 dx0 5 0.542635  f(x0 1 dx0) 5 0.038579

x1 5 0.537263 2 0.005373(0.047083)

0.038579 2 0.047083 5 0.56701  Zet Z 5 0.0236%

Third iteration:

x0 5 0.56701   f(x0) 5 0.000209

x0 1 dx0 5 0.572680  f(x0 1 dx0) 5 20.00867

x1 5 0.56701 2 0.00567(0.000209)

20.00867 2 0.000209 5 0.567143  Zet Z 5 2.365 3 10

25%

cha9792x_ch06_145-175.indd Page 161 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 161 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

162 OPEN METHODS

The choice of a proper value for � is not automatic. If � is too small, the method can be swamped by round-off error caused by subtractive cancellation in the denomina- tor of Eq. (6.8). If it is too big, the technique can become ineffi cient and even divergent. However, if chosen correctly, it provides a nice alternative for cases where evaluating the derivative is diffi cult and developing two initial guesses is inconvenient.

6.4 BRENT’S METHOD

Wouldn’t it be nice to have a hybrid approach that combined the reliability of bracketing with the speed of the open methods? Brent’s root-location method is a clever algorithm that does just that by applying a speedy open method wherever possible, but reverting to a reliable bracketing method if necessary. The approach was developed by Richard Brent (1973) based on an earlier algorithm of Theodorus Dekker (1969). The bracketing technique is the trusty bisection method (Sec. 5.2) whereas two differ- ent open methods are employed. The fi rst is the secant method described in Sec. 6.3. As explained next, the second is inverse quadratic interpolation.

6.4.1 Inverse Quadratic Interpolation

Inverse quadratic interpolation is similar in spirit to the secant method. As in Fig. 6.10a, the secant method is based on computing a straight line that goes through two guesses. The intersection of this straight line with the x axis represents the new root estimate. For this reason, it is sometimes referred to as a linear interpolation method. Now suppose that we had three points. In that case, we could determine a quadratic function of x that goes through the three points (Fig. 6.10b). Just as with the linear secant method, the intersection of this parabola with the x axis would represent the new root estimate. And as illustrated in Fig. 6.10b, using a curve rather than a straight line often yields a better estimate. Although this would seem to represent a great improvement, the approach has a fundamental fl aw: It is possible that the parabola might not intersect the x axis! Such would be the case when the resulting parabola had complex roots. This is illustrated by the parabola, y 5 f(x), in Fig. 6.11.

FIGURE 6.10 Comparison of (a) the secant method and (b) inverse qua- dratic interpolation. Note that the dark parabola passing through the three points in (b) is called “inverse” because it is written in y rather than in x.

f (x)

x

(a) (b)

f (x)

x

cha9792x_ch06_145-175.indd Page 162 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 162 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.4 BRENT’S METHOD 163

The diffi culty can be rectifi ed by employing inverse quadratic interpolation. That is, rather than using a parabola in x, we can fi t the points with a parabola in y. This amounts to reversing the axes and creating a “sideways” parabola [the curve, x 5 f(y), in Fig. 6.11]. If the three points are designated as (xi22, yi22), (xi21, yi21), and (xi, yi), a quadratic function of y that passes through the points can be generated as

g(y) 5 (y 2 yi21) (y 2 yi)

(yi22 2 yi21) (yi22 2 yi) xi22 1

(y 2 yi22) (y 2 yi)

(yi21 2 yi22) (yi21 2 yi) xi21

1 (y 2 yi22) (y 2 yi21)

(yi 2 yi22) (yi 2 yi21) xi (6.9)

As we will learn in Sec. 18.2, this form is called a Lagrange polynomial. The root, xi11, corresponds to y 5 0, which when substituted into Eq. (6.9) yields

xi11 5 yi21 yi

(yi22 2 yi21) (yi22 2 yi) xi22 1

yi22 yi (yi21 2 yi21 2 yi)

xi21

1 yi22 yi21

(yi 2 yi22) (yi 2 yi21) xi (6.10)

As shown in Fig. 6.11, such a “sideways” parabola always intersects the x axis.

EXAMPLE 6.9 Inverse Quadratic Interpolation

Problem Statement. Develop quadratic equations in both x and y for the data points depicted in Fig. 6.11: (1, 2), (2, 1), and (4, 5). For the fi rst, y 5 f(x), employ the qua- dratic formula to illustrate that the roots are complex. For the latter, x 5 g(y), use inverse quadratic interpolation (Eq. 6.10) to determine the root estimate.

FIGURE 6.11 Two parabolas fi t to three points. The parabola written as a function of x, y 5 f (x), has complex roots and hence does not intersect the x axis. In contrast, if the variables are reversed, and the parabola developed as x 5 f (y), the function does intersect the x axis.

5

Root

31 20

2

4

6

y

x = f (y)

y = f (x)

x

cha9792x_ch06_145-175.indd Page 163 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 163 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

164 OPEN METHODS

Solution. By reversing the x’s and y’s, Eq. (6.9) can be used to generate a quadratic in x as

f(x) 5 (x 2 2)(x 2 4)

(1 2 2)(1 2 4) 2 1

(x 2 1)(x 2 4)

(2 2 1)(2 2 4) 1 1

(x 2 1)(x 2 2)

(4 2 1)(4 2 2) 5

or collecting terms

f(x) 5 x2 2 4x 1 5

This equation was used to generate the parabola, y 5 f(x), in Fig. 6.11. The quadratic formula can be used to determine that the roots for this case are complex,

x 5 4 6 2(24)2 2 4(1)(5)

2 5 2 6 i

Equation (6.9) can be used to generate the quadratic in y as

g(y) 5 (y 2 1)(y 2 5)

(2 2 1)(2 2 5) 1 1

(y 2 2)(y 2 5)

(1 2 2)(1 2 5) 2 1

(y 2 2)(y 2 1)

(5 2 2)(5 2 1) 4

or collecting terms

g(y) 5 0.5x2 2 2.5x 1 4

Finally, Eq. (6.10) can be used to determine the root as

xi11 5 21(25)

(2 2 1)(2 2 5) 1 1

22(25)

(1 2 2)(1 2 5) 2 1

22(21)

(5 2 2)(5 2 1) 4 5 4

Before proceeding to Brent’s algorithm, we need to mention one more case where inverse quadratic interpolation does not work. If the three y values are not distinct (that is, yi22 5 yi21 or yi21 5 yi), an inverse quadratic function does not exist. So this is where the secant method comes into play. If we arrive at a situation where the y values are not distinct, we can always revert to the less effi cient secant method to generate a root using two of the points. If yi22 5 yi21, we use the secant method with xi21 and xi. If yi21 5 yi, we use xi22 and xi21.

6.4.2 Brent’s Method Algorithm

The general idea behind the Brent’s root fi nding method is whenever possible to use one of the quick open methods. In the event that these generate an unacceptable result (i.e., a root estimate that falls outside the bracket), the algorithm reverts to the more conservative bisection method. Although bisection may be slower, it generates an estimate guaranteed to fall within the bracket. This process is then repeated until the root is located to within an acceptable tolerance. As might be expected, bisection typically dominates at fi rst but as the root is approached, the technique shifts to the faster open methods. Figure 6.12 presents pseudocode for the algorithm based on a MATLAB software M-fi le developed by Cleve Moler (2005). It represents a stripped down version of

cha9792x_ch06_145-175.indd Page 164 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 164 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.4 BRENT’S METHOD 165

Function fzerosimp(xl, xu) eps 5 2.22044604925031E-16 tol 5 0.000001 a 5 xl: b 5 xu: fa 5 f(a): fb 5 f(b) c 5 a: fc 5 fa: d 5 b 2 c: e 5 d DO IF fb 5 0 EXIT IF Sgn(fa) 5 Sgn(fb) THEN (If necessary, rearrange points) a 5 c: fa 5 fc: d 5 b 2 c: e 5 d ENDIF IF |fa| , |fb| THEN c 5 b: b 5 a: a 5 c fc 5 fb: fb 5 fa: fa 5 fc ENDIF m 5 0.5 * (a 2 b) (Termination test and possible exit) tol 5 2 * eps * max(|b|, 1) IF |m| # tol Or fb 5 0. THEN EXIT ENDIF (Choose open methods or bisection) IF |e| $ tol And |fc| . |fb| THEN s 5 fb / fc IF a 5 c THEN (Secant method) p 5 2 * m * s q 5 1 2 s ELSE (Inverse quadratic interpolation) q 5 fc / fa: r 5 fb / fa p 5 s * (2 * m * q * (q 2 r) 2 (b 2 c) * (r 2 1)) q 5 (q 2 1) * (r 2 1) * (s 2 1) ENDIF IF p . 0 THEN q 5 2q ELSE p 5 2p IF 2 * p , 3 * m * q 2 |tol * q| AND p , |0.5 * e * q| THEN e 5 d: d 5 p / q ELSE d 5 m: e 5 m ENDIF ELSE (Bisection) d 5 m: e 5 m ENDIF c 5 b: fc 5 fb IF |d| . tol THEN b 5 b 1 d Else b 5 b 2 Sgn(b 2 a) * tol fb 5 f(b) ENDDO fzerosimp 5 b END fzerosimp

FIGURE 6.12 Pseudocode for Brent’s root fi nding algorithm based on a MATLAB m-fi le developed by Cleve Moler (2005).

cha9792x_ch06_145-175.indd Page 165 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 165 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

166 OPEN METHODS

the fzero function which is the professional root location function employed in MAT- LAB. For that reason, we call the simplifi ed version: fzerosimp. Note that it requires another function, f, that holds the equation for which the root is being evaluated. The fzerosimp function is passed two initial guesses that must bracket the root. After assigning values for machine epsilon and a tolerance, the three variables defi ning the search interval (a, b, c) are initialized, and f is evaluated at the endpoints. A main loop is then implemented. If necessary, the three points are rearranged to satisfy the conditions required for the algorithm to work effectively. At this point, if the stopping criteria are met, the loop is terminated. Otherwise, a decision structure chooses among the three methods and checks whether the outcome is acceptable. A fi nal section then evaluates f at the new point and the loop is repeated. Once the stopping criteria are met, the loop terminates and the fi nal root estimate is returned. Note that Sec. 7.7.2 presents an application of Brent’s method where we illustrate how the MATLAB’s fzero function works. In addition, it is employed in Case Study 8.4 to determine the friction factor for air fl ow through a tube.

6.5 MULTIPLE ROOTS

A multiple root corresponds to a point where a function is tangent to the x axis. For example, a double root results from

f(x) 5 (x 2 3)(x 2 1)(x 2 1) (6.11)

or, multiplying terms, f(x) 5 x3 2 5x2 1 7x 2 3. The equation has a double root because one value of x makes two terms in Eq. (6.11) equal to zero. Graphically, this corresponds to the curve touching the x axis tangentially at the double root. Examine Fig. 6.13a at x 5 1. Notice that the function touches the axis but does not cross it at the root. A triple root corresponds to the case where one x value makes three terms in an equation equal to zero, as in

f(x) 5 (x 2 3)(x 2 1)(x 2 1)(x 2 1)

or, multiplying terms, f(x) 5 x4 2 6x3 1 12x2 2 10x 1 3. Notice that the graphical depiction (Fig. 6.13b) again indicates that the function is tangent to the axis at the root, but that for this case the axis is crossed. In general, odd multiple roots cross the axis, whereas even ones do not. For example, the quadruple root in Fig. 6.13c does not cross the axis. Multiple roots pose some diffi culties for many of the numerical methods described in Part Two:

1. The fact that the function does not change sign at even multiple roots precludes the use of the reliable bracketing methods that were discussed in Chap. 5. Thus, of the methods covered in this book, you are limited to the open methods that may diverge.

2. Another possible problem is related to the fact that not only f(x) but also f9(x) goes to zero at the root. This poses problems for both the Newton-Raphson and secant methods, which both contain the derivative (or its estimate) in the denominator of

cha9792x_ch06_145-175.indd Page 166 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 166 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.5 MULTIPLE ROOTS 167

their respective formulas. This could result in division by zero when the solution converges very close to the root. A simple way to circumvent these problems is based on the fact that it can be demonstrated theoretically (Ralston and Rabinowitz, 1978) that f(x) will always reach zero before f9(x). Therefore, if a zero check for f(x) is incorporated into the computer program, the computation can be terminated before f9(x) reaches zero.

3. It can be demonstrated that the Newton-Raphson and secant methods are linearly, rather than quadratically, convergent for multiple roots (Ralston and Rabinowitz, 1978). Modifi cations have been proposed to alleviate this problem. Ralston and Rabinowitz (1978) have indicated that a slight change in the formulation returns it to quadratic convergence, as in

xi11 5 xi 2 m f(xi)

f ¿(xi) (6.12)

where m is the multiplicity of the root (that is, m 5 2 for a double root, m 5 3 for a triple root, etc.). Of course, this may be an unsatisfactory alternative because it hinges on foreknowledge of the multiplicity of the root.

Another alternative, also suggested by Ralston and Rabinowitz (1978), is to defi ne a new function u(x), that is, the ratio of the function to its derivative, as in

u(x) 5 f  (x)

f ¿(x) (6.13)

It can be shown that this function has roots at all the same locations as the original function. Therefore, Eq. (6.13) can be substituted into Eq. (6.6) to develop an alternative form of the Newton-Raphson method:

xi11 5 xi 2 u(xi)

u¿(xi) (6.14)

Equation (6.13) can be differentiated to give

u¿(x) 5 f ¿(x) f ¿(x) 2 f(x) f –(x)

[ f ¿(x)]2 (6.15)

Equations (6.13) and (6.15) can be substituted into Eq. (6.14) and the result simplifi ed to yield

xi11 5 xi 2 f(xi) f ¿(xi)

[ f ¿(xi)]2 2 f(xi) f –(xi) (6.16)

EXAMPLE 6.10 Modifi ed Newton-Raphson Method for Multiple Roots

Problem Statement. Use both the standard and modifi ed Newton-Raphson methods to evaluate the multiple root of Eq. (6.11), with an initial guess of x0 5 0.

FIGURE 6.13 Examples of multiple roots that are tangential to the x axis. Notice that the function does not cross the axis on either side of even multiple roots (a) and (c), whereas it crosses the axis for odd cases (b).

f (x)

x

(a)

Double root

1 3

4

0

– 4

f (x)

x

(c)

Quadruple root

1 3

4

0

– 4

f (x)

x

(b)

Triple root

1 3

4

0

– 4

cha9792x_ch06_145-175.indd Page 167 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 167 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

168 OPEN METHODS

Solution. The fi rst derivative of Eq. (6.11) is f ¿(x) 5 3x2 2 10x 1 7, and therefore, the standard Newton-Raphson method for this problem is [Eq. (6.6)]

xi11 5 xi 2 x3i 2 5x

2 i 1 7xi 2 3

3x2i 2 10xi 1 7

which can be solved iteratively for

i xi et (%)

0 0 100 1 0.4285714 57 2 0.6857143 31 3 0.8328654 17 4 0.9133290 8.7 5 0.9557833 4.4 6 0.9776551 2.2

As anticipated, the method is linearly convergent toward the true value of 1.0. For the modifi ed method, the second derivative is f 0(x) 5 6x 2 10, and the iterative relationship is [Eq. (6.16)]

xi11 5 xi 2 (x3i 2 5x

2 i 1 7xi 2 3)(3x

2 i 2 10xi 1 7)

(3x2i 2 10xi 1 7) 2 2 (x3i 2 5x

2 i 1 7xi 2 3)(6xi 2 10)

which can be solved for

i xi et (%)

0 0 100 1 1.105263 11 2 1.003082 0.31 3 1.000002 0.00024

Thus, the modifi ed formula is quadratically convergent. We can also use both methods to search for the single root at x 5 3. Using an initial guess of x0 5 4 gives the following results:

i Standard et (%) Modifi ed et (%)

0 4 33 4 33 1 3.4 13 2.636364 12 2 3.1 3.3 2.820225 6.0 3 3.008696 0.29 2.961728 1.3 4 3.000075 0.0025 2.998479 0.051 5 3.000000 2 3 1027 2.999998 7.7 3 1025

Thus, both methods converge quickly, with the standard method being somewhat more effi cient.

cha9792x_ch06_145-175.indd Page 168 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 168 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.6 SYSTEMS OF NONLINEAR EQUATIONS 169

The preceding example illustrates the trade-offs involved in opting for the modifi ed Newton-Raphson method. Although it is preferable for multiple roots, it is somewhat less effi cient and requires more computational effort than the standard method for simple roots. It should be noted that a modifi ed version of the secant method suited for multiple roots can also be developed by substituting Eq. (6.13) into Eq. (6.7). The resulting formula is (Ralston and Rabinowitz, 1978)

xi11 5 xi 2 u(xi)(xi21 2 xi)

u(xi21) 2 u(xi)

6.6 SYSTEMS OF NONLINEAR EQUATIONS

To this point, we have focused on the determination of the roots of a single equation. A related problem is to locate the roots of a set of simultaneous equations,

f1(x1, x2, p , xn) 5 0 f2(x1, x2, p , xn) 5 0 � �

� � (6.17)

� �

fn(x1, x2, p , xn) 5 0

The solution of this system consists of a set of x values that simultaneously result in all the equations equaling zero. In Part Three, we will present methods for the case where the simultaneous equations are linear—that is, they can be expressed in the general form

f(x) 5 a1x1 1 a2x2 1 p 1 anxn 2 b 5 0 (6.18)

where the b and the a’s are constants. Algebraic and transcendental equations that do not fi t this format are called nonlinear equations. For example,

x2 1 xy 5 10

and

y 1 3xy2 5 57

are two simultaneous nonlinear equations with two unknowns, x and y. They can be expressed in the form of Eq. (6.17) as

u(x, y) 5 x2 1 xy 2 10 5 0 (6.19a)

y(x, y) 5 y 1 3xy2 2 57 5 0 (6.19b)

Thus, the solution would be the values of x and y that make the functions u(x, y) and y(x, y) equal to zero. Most approaches for determining such solutions are extensions of the open methods for solving single equations. In this section, we will investigate two of these: fi xed-point iteration and Newton-Raphson.

cha9792x_ch06_145-175.indd Page 169 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 169 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

170 OPEN METHODS

6.6.1 Fixed-Point Iteration

The fi xed-point-iteration approach (Sec. 6.1) can be modifi ed to solve two simultaneous, nonlinear equations. This approach will be illustrated in the following example.

EXAMPLE 6.11 Fixed-Point Iteration for a Nonlinear System

Problem Statement. Use fi xed-point iteration to determine the roots of Eq. (6.19). Note that a correct pair of roots is x 5 2 and y 5 3. Initiate the computation with guesses of x 5 1.5 and y 5 3.5.

Solution. Equation (6.19a) can be solved for

xi11 5 10 2 x2i

yi (E6.11.1)

and Eq. (6.19b) can be solved for

yi11 5 57 2 3xi y 2 i (E6.11.2)

Note that we will drop the subscripts for the remainder of the example. On the basis of the initial guesses, Eq. (E6.11.1) can be used to determine a new value of x:

x 5 10 2 (1.5)2

3.5 5 2.21429

This result and the initial value of y 5 3.5 can be substituted into Eq. (E6.11.2) to determine a new value of y:

y 5 57 2 3(2.21429)(3.5)2 5 224.37516

Thus, the approach seems to be diverging. This behavior is even more pronounced on the second iteration:

x 5 10 2 (2.21429)2

224.37516 5 20.20910

y 5 57 2 3(20.20910)(224.37516)2 5 429.709

Obviously, the approach is deteriorating. Now we will repeat the computation but with the original equations set up in a different format. For example, an alternative formulation of Eq. (6.19a) is

x 5 210 2 xy

and of Eq. (6.19b) is

y 5 B

57 2 y

3x

Now the results are more satisfactory:

x 5 210 2 1.5(3.5) 5 2.17945

cha9792x_ch06_145-175.indd Page 170 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 170 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

6.6 SYSTEMS OF NONLINEAR EQUATIONS 171

y 5 B

57 2 3.5

3(2.17945) 5 2.86051

x 5 210 2 2.17945(2.86051) 5 1.94053

y 5 B

57 2 2.86051

3(1.94053) 5 3.04955

Thus, the approach is converging on the true values of x 5 2 and y 5 3.

The previous example illustrates the most serious shortcoming of simple fi xed-point iteration—that is, convergence often depends on the manner in which the equations are formulated. Additionally, even in those instances where convergence is possible, diver- gence can occur if the initial guesses are insuffi ciently close to the true solution. Using reasoning similar to that in Box 6.1, it can be demonstrated that suffi cient conditions for convergence for the two-equation case are

` 0u 0x ` 1 ` 0u

0y ` , 1

and

` 0y 0x ` 1 ` 0y

0y ` , 1

These criteria are so restrictive that fi xed-point iteration has limited utility for solving nonlinear systems. However, as we will describe later in the book, it can be very useful for solving linear systems.

6.6.2 Newton-Raphson

Recall that the Newton-Raphson method was predicated on employing the derivative (that is, the slope) of a function to estimate its intercept with the axis of the independent variable—that is, the root (Fig. 6.5). This estimate was based on a fi rst-order Taylor series expansion (recall Box 6.2),

f(xi11) 5 f(xi) 1 (xi11 2 xi) f ¿(xi) (6.20)

where xi is the initial guess at the root and xi11 is the point at which the slope intercepts the x axis. At this intercept, f(xi11) by defi nition equals zero and Eq. (6.20) can be rear- ranged to yield

xi11 5 xi 2 f(xi)

f ¿(xi) (6.21)

which is the single-equation form of the Newton-Raphson method. The multiequation form is derived in an identical fashion. However, a multivariable Taylor series must be used to account for the fact that more than one independent

cha9792x_ch06_145-175.indd Page 171 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 171 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

172 OPEN METHODS

variable contributes to the determination of the root. For the two-variable case, a fi rst- order Taylor series can be written [recall Eq. (4.26)] for each nonlinear equation as

ui11 5 ui 1 (xi11 2 xi) 0ui 0x

1 (yi11 2 yi) 0ui 0y

(6.22a)

and

yi11 5 yi 1 (xi11 2 xi) 0yi 0x

1 (yi11 2 yi) 0yi 0y

(6.22b)

Just as for the single-equation version, the root estimate corresponds to the values of x and y, where ui11 and yi11 equal zero. For this situation, Eq. (6.22) can be rearranged to give

0ui 0x

xi11 1 0ui 0y

yi11 5 2ui 1 xi 0ui 0x

1 yi 0ui 0y

(6.23a)

0yi 0x

xi11 1 0yi 0y

yi11 5 2yi 1 xi 0yi 0x

1 yi 0yi 0y

(6.23b)

Because all values subscripted with i’s are known (they correspond to the latest guess or approximation), the only unknowns are xi11 and yi11. Thus, Eq. (6.23) is a set of two linear equations with two unknowns [compare with Eq. (6.18)]. Consequently, algebraic manipulations (for example, Cramer’s rule) can be employed to solve for

xi11 5 xi 2

ui 0yi 0y

2 yi 0ui 0y

0ui 0x

0yi 0y

2 0ui 0y

0yi 0x

(6.24a)

yi11 5 yi 2

yi 0ui 0x

2 ui 0yi 0x

0ui 0x

0yi 0y

2 0ui 0y

0yi 0x

(6.24b)

The denominator of each of these equations is formally referred to as the determinant of the Jacobian of the system. Equation (6.24) is the two-equation version of the Newton-Raphson method. As in the following example, it can be employed iteratively to home in on the roots of two simultaneous equations.

EXAMPLE 6.12 Newton-Raphson for a Nonlinear System

Problem Statement. Use the multiple-equation Newton-Raphson method to determine roots of Eq. (6.19). Note that a correct pair of roots is x 5 2 and y 5 3. Initiate the computation with guesses of x 5 1.5 and y 5 3.5.

Solution. First compute the partial derivatives and evaluate them at the initial guesses of x and y:

0u0 0x

5 2x 1 y 5 2(1.5) 1 3.5 5 6.5   0u0 0y

5 x 5 1.5

cha9792x_ch06_145-175.indd Page 172 15/10/13 7:12 PM f-479 cha9792x_ch06_145-175.indd Page 172 15/10/13 7:12 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 173

0y0 0x

5 3y2 5 3(3.5)2 5 36.75     0y0 0y

5 1 1 6xy 5 1 1 6(1.5)(3.5) 5 32.5

Thus, the determinant of the Jacobian for the fi rst iteration is

6.5(32.5) 2 1.5(36.75) 5 156.125

The values of the functions can be evaluated at the initial guesses as

u0 5 (1.5) 2 1 1.5(3.5) 2 10 5 22.5

y0 5 3.5 1 3(1.5)(3.5) 2 2 57 5 1.625

These values can be substituted into Eq. (6.24) to give

x 5 1.5 2 22.5(32.5) 2 1.625(1.5)

156.125 5 2.03603

y 5 3.5 2 1.625(6.5) 2 (22.5)(36.75)

156.125 5 2.84388

Thus, the results are converging to the true values of x 5 2 and y 5 3. The computation can be repeated until an acceptable accuracy is obtained.

Just as with fi xed-point iteration, the Newton-Raphson approach will often diverge if the initial guesses are not suffi ciently close to the true roots. Whereas graphical methods could be employed to derive good guesses for the single-equation case, no such simple procedure is available for the multiequation version. Although there are some advanced approaches for obtaining acceptable fi rst estimates, often the initial guesses must be ob- tained on the basis of trial and error and knowledge of the physical system being modeled. The two-equation Newton-Raphson approach can be generalized to solve n simulta- neous equations. Because the most effi cient way to do this involves matrix algebra and the solution of simultaneous linear equations, we will defer discussion of the general approach to Part Three.

PROBLEMS

6.1 Use simple fi xed-point iteration to locate the root of

f (x) 5 sin (1x) 2 x

Use an initial guess of x0 5 0.5 and iterate until ea # 0.01%. Verify that the process is linearly convergent as described in Box 6.1. 6.2 Determine the highest real root of

f (x) 5 2x3 2 11.7x2 1 17.7x 2 5

(a) Graphically. (b) Fixed-point iteration method (three iterations, x0 5 3). Note: Make

certain that you develop a solution that converges on the root. (c) Newton-Raphson method (three iterations, x0 5 3).

(d) Secant method (three iterations, x21 5 3, x0 5 4). (e) Modifi ed secant method (three iterations, x0 5 3, d 5 0.01). Compute the approximate percent relative errors for your solutions. 6.3 Use (a) fi xed-point iteration and (b) the Newton-Raphson method to determine a root of f(x) 5 20.9x2 1 1.7x 1 2.5 using x0 5 5. Perform the computation until ea is less than es 5 0.01%. Also perform an error check of your fi nal answer. 6.4 Determine the real roots of f (x) 5 21 1 5.5x 2 4x2 1 0.5x3: (a) graphically and (b) using the Newton-Raphson method to within es 5 0.01%. 6.5 Employ the Newton-Raphson method to determine a real root for f (x) 5 21 1 5.5x 2 4x2 1 0.5x3 using initial guesses of (a) 4.52

cha9792x_ch06_145-175.indd Page 173 15/10/13 7:13 PM f-479 cha9792x_ch06_145-175.indd Page 173 15/10/13 7:13 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

174 OPEN METHODS

and (b) 4.54. Discuss and use graphical and analytical methods to ex- plain any peculiarities in your results. 6.6 Determine the lowest real root of f (x) 5 212 2 21x 1 18x2 2 2.4x3: (a) graphically and (b) using the secant method to a value of es corresponding to three signifi cant fi gures. 6.7 Locate the fi rst positive root of

f (x) 5 sin x 1 cos (1 1 x2) 2 1

where x is in radians. Use four iterations of the secant method with initial guesses of (a) xi21 5 1.0 and xi 5 3.0; (b) xi21 5 1.5 and xi 5 2.5, and (c) xi21 5 1.5 and xi 5 2.25 to locate the root. (d) Use the graphical method to explain your results. 6.8 Determine the real root of x3.5 5 80, with the modifi ed secant method to within es 5 0.1% using an initial guess of x0 5 3.5 and d 5 0.01. 6.9 Determine the highest real root of f(x) 5 x3 2 6x2 1 11x 2 6.1: (a) Graphically. (b) Using the Newton-Raphson method (three iterations, xi 5 3.5). (c) Using the secant method (three iterations, xi11 5 2.5 and

xi 5 3.5). (d) Using the modifi ed secant method (three iterations, xi 5 3.5, d 5 0.01).

6.10 Determine the lowest positive root of f (x) 5 7 sin (x)e2x 2 1: (a) Graphically. (b) Using the Newton-Raphson method (three iterations, xi 5 0.3). (c) Using the secant method (fi ve iterations, xi21 5 0.5 and

xi 5 0.4). (d) Using the modifi ed secant method (three iterations, xi 5 0.3, d 5 0.01).

6.11 Use the Newton-Raphson method to fi nd the root of

f (x) 5 e20.5x(4 2 x) 2 2

Employ initial guesses of (a) 2, (b) 6, and (c) 8. Explain your results. 6.12 Given

f (x) 5 22x6 2 1.5x4 1 10x 1 2

Use a root location technique to determine the maximum of this function. Perform iterations until the approximate relative error falls below 5%. If you use a bracketing method, use initial guesses of xl 5 0 and xu 5 1. If you use the Newton-Raphson or the modi- fi ed secant method, use an initial guess of xi 5 1. If you use the secant method, use initial guesses of xi21 5 0 and xi 5 1. Assuming that convergence is not an issue, choose the technique that is best suited to this problem. Justify your choice. 6.13 You must determine the root of the following easily differen- tiable function,

e0.5x 5 5 2 5x

Pick the best numerical technique, justify your choice and then use that technique to determine the root. Note that it is known that for positive initial guesses, all techniques except fi xed-point iteration will eventually converge. Perform iterations until the approximate relative error falls below 2%. If you use a bracket- ing method, use initial guesses of xl 5 0 and xu 5 2. If you use the Newton-Raphson or the modifi ed secant method, use an ini- tial guess of xi 5 0.7. If you use the secant method, use initial guesses of xi21 5 0 and xi 5 2. 6.14 Use (a) the Newton-Raphson method and (b) the modifi ed secant method (d 5 0.05) to determine a root of f(x) 5 x5 2 16.05x4 1 88.75x3 2 192.0375x2 1 116.35x 1 31.6875 using an initial guess of x 5 0.5825 and es 5 0.01%. Explain your results. 6.15 The “divide and average” method, an old-time method for approximating the square root of any positive number a, can be formulated as

x 5 x 1 ayx

2

Prove that this is equivalent to the Newton-Raphson algorithm. 6.16 (a) Apply the Newton-Raphson method to the function f(x) 5 tanh(x2 2 9) to evaluate its known real root at x 5 3. Use an initial guess of x0 5 3.2 and take a minimum of four iterations. (b) Did the method exhibit convergence onto its real root? Sketch the plot with the results for each iteration shown. 6.17 The polynomial f(x) 5 0.0074x4 2 0.284x3 1 3.355x2 2 12.183x 1 5 has a real root between 15 and 20. Apply the Newton- Raphson method to this function using an initial guess of x0 5 16.15. Explain your results. 6.18 Use the secant method on the circle function (x 1 1)2 1 (y 2 2)2 5 16 to fi nd a positive real root. Set your initial guess to xi 5 3 and xi21 5 0.5. Approach the solution from the fi rst and fourth quadrants. When solving for f(x) in the fourth quadrant, be sure to take the negative value of the square root. Why does your solution diverge? 6.19 You are designing a spherical tank (Fig. P6.19) to hold water for a small village in a developing country. The volume of liquid it can hold can be computed as

V 5 ph2 [3R 2 h]

3

where V 5 volume (m3), h 5 depth of water in tank (m), and R 5 the tank radius (m). If R 5 3 m, what depth must the tank be fi lled to so that it holds 30 m3? Use three iterations of the Newton- Raphson method to determine your answer. Determine the ap- proximate relative error after each iteration. Note that an initial guess of R will always converge.

cha9792x_ch06_145-175.indd Page 174 15/10/13 7:13 PM f-479 cha9792x_ch06_145-175.indd Page 174 15/10/13 7:13 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 175

6.20 The Manning equation can be written for a rectangular open channel as

Q 5 1S(BH)5y3

n(B 1 2H)2y3

where Q 5 fl ow [m3/s], S 5 slope [m/m], H 5 depth [m], and n 5 the Manning roughness coeffi cient. Develop a fi xed-point iteration scheme to solve this equation for H given Q 5 5, S 5 0.0002, B 5 20, and n 5 0.03. Prove that your scheme converges for all initial guesses greater than or equal to zero. 6.21 The function x3 2 2x2 2 4x 1 8 has a double root at x 5 2. Use (a) the standard Newton-Raphson [Eq. (6.6)], (b) the modi- fi ed Newton-Raphson [Eq. (6.12)], and (c) the modifi ed Newton- Raphson [Eq. (6.16)] to solve for the root at x 5 2. Compare and discuss the rate of convergence using an initial guess of x0 5 1.2. 6.22 Determine the roots of the following simultaneous nonlinear equations using (a) fi xed-point iteration and (b) the Newton-Raphson method:

y 5 2x2 1 x 1 0.75

y 1 5xy 5 x2

Employ initial guesses of x 5 y 5 1.2 and discuss the results. 6.23 Determine the roots of the simultaneous nonlinear equations

(x 2 4)2 1 (y 2 4)2 5 5

x2 1 y2 5 16

Use a graphical approach to obtain your initial guesses. Determine refi ned estimates with the two-equation Newton-Raphson method described in Sec. 6.6.2. 6.24 Repeat Prob. 6.23 except determine the positive root of

y 5 x2 1 1

y 5 2 cos x

6.25 A mass balance for a pollutant in a well-mixed lake can be written as

V dc

dt 5 W 2 Qc 2 kV 1c

Given the parameter values V 5 1 3 106m3, Q 5 1 3 105 m3/yr, W 5 1 3 106 g/yr, and k 5 0.25 m0.5/g0.5/yr, use the modifi ed secant method to solve for the steady-state concentration. Employ an ini- tial guess of c 5 4 g/m3 and d 5 0.5. Perform three iterations and determine the percent relative error after the third iteration. 6.26 For Prob. 6.25, the root can be located with fixed-point iteration as

c 5 aW 2 Qc kV

b2

or as

c 5 W 2 kV 1c

Q

Only one will converge for initial guesses of 2 , c , 6. Select the correct one and demonstrate why it will always work. 6.27 Develop a user-friendly program for the Newton-Raphson method based on Fig. 6.4 and Sec. 6.2.3. Test it by duplicating the computation from Example 6.3. 6.28 Develop a user-friendly program for the secant method based on Fig. 6.4 and Sec. 6.3.2. Test it by duplicating the computation from Example 6.6. 6.29 Develop a user-friendly program for the modifi ed secant method based on Fig. 6.4 and Sec. 6.3.2. Test it by duplicating the computation from Example 6.8. 6.30 Develop a user-friendly program for Brent’s root location method based on Fig. 6.12. Test it by solving Prob. 6.6. 6.31 Develop a user-friendly program for the two-equation Newton-Raphson method based on Sec. 6.6.2. Test it by solving Example 6.12. 6.32 Use the program you developed in Prob. 6.31 to solve Probs. 6.22 and 6.23 to within a tolerance of es 5 0.01%.

hV

R

FIGURE P6.19

cha9792x_ch06_145-175.indd Page 175 15/10/13 7:13 PM f-479 cha9792x_ch06_145-175.indd Page 175 15/10/13 7:13 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7 C H A P T E R 7

176

Roots of Polynomials

In this chapter, we will discuss methods to fi nd the roots of polynomial equations of the general form

fn(x) 5 a0 1 a1x 1 a2x 2 1 p 1 anxn (7.1)

where n 5 the order of the polynomial and the a’s 5 constant coeffi cients. Although the coeffi cients can be complex numbers, we will limit our discussion to cases where they are real. For such cases, the roots can be real and/or complex. The roots of such polynomials follow these rules:

1. For an nth-order equation, there are n real or complex roots. It should be noted that these roots will not necessarily be distinct.

2. If n is odd, there is at least one real root. 3. If complex roots exist, they exist in conjugate pairs (that is, l 1 mi and l 2 mi),

where i 5 121.

Before describing the techniques for locating the roots of polynomials, we will provide some background. The fi rst section offers some motivation for studying the techniques; the second deals with some fundamental computer manipulations involving polynomials.

7.1 POLYNOMIALS IN ENGINEERING AND SCIENCE

Polynomials have many applications in engineering and science. For example, they are used extensively in curve-fi tting. However, we believe that one of their most interesting and powerful applications is in characterizing dynamic systems and, in particular, linear systems. Examples include mechanical devices, structures, and electrical circuits. We will be explor- ing specifi c examples throughout the remainder of this text. In particular, they will be the focus of several of the engineering applications throughout the remainder of this text. For the time being, we will keep the discussion simple and general by focusing on a simple second-order system defi ned by the following linear ordinary differential equa- tion (or ODE):

a2 d2y

dt2 1 a1

dy

dt 1 a0y 5 F(t) (7.2)

cha9792x_ch07_176-203.indd Page 176 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 176 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.1 POLYNOMIALS IN ENGINEERING AND SCIENCE 177

where y and t are the dependent and independent variables, respectively, the a’s are constant coeffi cients, and F(t) is the forcing function. In addition, it should be noted that Eq. (7.2) can be alternatively expressed as a pair of fi rst-order ODEs by defi ning a new variable z,

z 5 dy

dt (7.3)

Equation (7.3) can be substituted along with its derivative into Eq. (7.2) to remove the second-derivative term. This reduces the problem to solving

dz

dt 5

F(t) 2 a1z 2 a0y a2

(7.4)

dz

dt 5 z (7.5)

In a similar fashion, an nth-order linear ODE can always be expressed as a system of n fi rst-order ODEs. Now let’s look at the solution. The forcing function represents the effect of the external world on the system. The homogeneous or general solution of the equation deals with the case when the forcing function is set to zero,

a2 d2y

dt2 1 a1

dy

dt 1 a0y 5 0 (7.6)

Thus, as the name implies, the general solution should tell us something very fundamental about the system being simulated—that is, how the system responds in the absence of external stimuli. Now, the general solution to all unforced linear systems is of the form y 5 ert. If this function is differentiated and substituted into Eq. (7.6), the result is

a2r 2ert 1 a1r e

rt 1 a0e rt 5 0

or canceling the exponential terms,

a2r 2 1 a1r 1 a0 5 0 (7.7)

Notice that the result is a polynomial called the characteristic equation. The roots of this polynomial are the values of r that satisfy Eq. (7.7). These r’s are referred to as the system’s characteristic values, or eigenvalues. So, here is the connection between roots of polynomials and engineering and science. The eigenvalue tells us something fundamental about the system we are modeling, and fi nding the eigenvalues involves fi nding the roots of polynomials. And, whereas fi nding the root of a second-order equation is easy with the quadratic formula, fi nding roots of higher-order systems (and hence, higher-order polynomials) is arduous analyti- cally. Thus, the best general approach requires numerical methods of the type described in this chapter. Before proceeding to these methods, let us take our analysis a bit farther by in- vestigating what specifi c values of the eigenvalues might imply about the behavior of

cha9792x_ch07_176-203.indd Page 177 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 177 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

178 ROOTS OF POLYNOMIALS

physical systems. First, let us evaluate the roots of Eq. (7.7) with the quadratic formula,

r1 r2

5 2a1 6 2a

2 1 2 4a2a0

a0

Thus, we get two roots. If the discriminant (a21 2 4a2a0) is positive, the roots are real and the general solution can be represented as

y 5 c1e r1t 1 c2e

r2t (7.8)

where the c’s 5 constants that can be determined from the initial conditions. This is called the overdamped case. If the discriminant is zero, a single real root results, and the general solution can be formulated as

y 5 (c1 1 c2t)e lt (7.9)

This is called the critically damped case. If the discriminant is negative, the roots will be complex conjugate numbers,

r1 r2

5 l 6 mi

and the general solution can be formulated as

y 5 c1e (l1mi) t 1 c2e

(l2mi) t

FIGURE 7.1 The general solution for linear ODEs can be composed of (a) exponential and (b) sinusoidal components. The combination of the two shapes results in the damped sinusoid shown in (c).

y

t(a) (b)

y

t

(c)

y

t

cha9792x_ch07_176-203.indd Page 178 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 178 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.2 COMPUTING WITH POLYNOMIALS 179

The physical behavior of this solution can be elucidated by using Euler’s formula

emit 5 cos mt 1 i sin mt

to reformulate the general solution as (see Boyce and DiPrima, 1992, for details of the derivation)

y 5 c1e lt cos mt 1 c2e

lt sin mt (7.10)

This is called the underdamped case. Equations (7.8), (7.9), and (7.10) express the possible ways that linear systems re- spond dynamically. The exponential terms mean that the solutions are capable of decay- ing (negative real part) or growing (positive real part) exponentially with time (Fig. 7.1a). The sinusoidal terms (imaginary part) mean that the solutions can oscillate (Fig. 7.1b). If the eigenvalue has both real and imaginary parts, the exponential and sinusoidal shapes are combined (Fig. 7.1c). Because such knowledge is a key element in understanding, designing, and controlling the behavior of a physical system, characteristic polynomials are very important in engineering and many branches of science. We will explore the dynamics of several engineering systems in the applications covered in Chap. 8.

7.2 COMPUTING WITH POLYNOMIALS

Before describing root-location methods, we will discuss some fundamental computer operations involving polynomials. These have utility in their own right as well as provid- ing support for root fi nding.

7.2.1 Polynomial Evaluation and Differentiation

Although it is the most common format, Eq. (7.1) provides a poor means for determin- ing the value of a polynomial for a particular value of x. For example, evaluating a third-order polynomial as

f3(x) 5 a3 x 3 1 a2 x

2 1 a1x 1 a0 (7.11)

involves six multiplications and three additions. In general, for an nth-order polynomial, this approach requires n(n 1 1)y2 multiplications and n additions. In contrast, a nested format,

f3(x) 5 ((a3x 1 a2)x 1 a1)x 1 a0 (7.12)

involves three multiplications and three additions. For an nth-order polynomial, this ap- proach requires n multiplications and n additions. Because the nested format minimizes the number of operations, it also tends to minimize round-off errors. Note that, depend- ing on your preference, the order of nesting can be reversed:

f3(x) 5 a0 1 x(a1 1 x(a2 1 xa3) ) (7.13)

Succinct pseudocode to implement the nested form can be written simply as

DOFOR j 5 n, 0, 21 p 5 p * x1a(j) END DO

cha9792x_ch07_176-203.indd Page 179 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 179 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

180 ROOTS OF POLYNOMIALS

where p holds the value of the polynomial (defi ned by its coeffi cients, the a’s) evaluated at x. There are cases (such as in the Newton-Raphson method) where you might want to evaluate both the function and its derivative. This evaluation can also be neatly included by adding a single line to the preceding pseudocode,

D0FOR j 5 n, 0, 21

df 5 df * x1p

p 5 p * x1a(j) END DO

where df holds the fi rst derivative of the polynomial.

7.2.2 Polynomial Defl ation

Suppose that you determine a single root of an nth-order polynomial. If you repeat your root location procedure, you might fi nd the same root. Therefore, it would be nice to remove the found root before proceeding. This removal process is referred to as polyno- mial defl ation. Before we show how this is done, some orientation might be useful. Polynomials are typically represented in the format of Eq. (7.1). For example, a fi fth-order polynomial could be written as

f5(x) 5 2120 2 46x 1 79x 2 2 3x3 2 7x4 1 x5 (7.14)

Although this is a familiar format, it is not necessarily the best expression to understand the polynomial’s mathematical behavior. For example, this fi fth-order polynomial might be expressed alternatively as

f5(x) 5 (x 1 1)(x 2 4)(x 2 5)(x 1 3)(x 2 2) (7.15)

This is called the factored form of the polynomial. If multiplication is completed and like terms collected, Eq. (7.14) would be obtained. However, the format of Eq. (7.15) has the advantage that it clearly indicates the function’s roots. Thus, it is apparent that x 5 21, 4, 5, 23, and 2 are all roots because each causes an individual term in Eq. (7.15) to become zero. Now, suppose that we divide this fi fth-order polynomial by any of its factors, for example, x 1 3. For this case, the result would be a fourth-order polynomial

f4(x) 5 (x 1 1)(x 2 4)(x 2 5)(x 2 2) 5 240 2 2x 1 27x 2 2 10x3 1 x4 (7.16)

with a remainder of zero. In the distant past, you probably learned to divide polynomials using the approach called synthetic division. Several computer algorithms (based on both synthetic division and other methods) are available for performing the operation. One simple scheme is provided by the following pseudocode, which divides an nth-order polynomial by a

cha9792x_ch07_176-203.indd Page 180 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 180 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.2 COMPUTING WITH POLYNOMIALS 181

monomial factor x 2 t:

r 5 a(n) a(n) 5 0 DOFOR i 5 n21, 0, 21

s 5 a(i) a(i) 5 r r 5 s 1 r * t END DO

If the monomial is a root of the polynomial, the remainder r will be zero, and the coef- fi cients of the quotient stored in a, at the end of the loop.

EXAMPLE 7.1 Polynomial Defl ation

Problem Statement. Divide the second-order polynomial,

f(x) 5 (x 2 4)(x 1 6) 5 x2 1 2x 2 24

by the factor x 2 4.

Solution. Using the approach outlined in the above pseudocode, the parameters are n 5 2, a0 5 224, a1 5 2, a2 5 1, and t 5 4. These can be used to compute

r 5 a2 5 1 a2 5 0

The loop is then iterated from i 5 2 2 1 5 1 to 0. For i 5 1,

s 5 a1 5 2 a1 5 r 5 1 r 5 s 1 rt 5 2 1 1(4) 5 6

For i 5 0,

s 5 a0 5 224 a0 5 r 5 6 r 5 224 1 6(4) 5 0

Thus, the result is as expected—the quotient is a0 1 a1x 5 6 1 x, with a remainder of zero.

It is also possible to divide by polynomials of higher order. As we will see later in this chapter, the most common task involves dividing by a second-order polynomial or parabola. The subroutine in Fig. 7.2 addresses the more general problem of dividing an nth-order polynomial a by an mth-order polynomial d. The result is an (n 2 m)th-order polynomial q, with an (m 2 1)th-order polynomial as the remainder. Because each calculated root is known only approximately, it should be noted that defl ation is sensitive to round-off errors. In some cases, round-off error can grow to the point that the results can become meaningless. Some general strategies can be applied to minimize this problem. For example, round-off error is affected by the order in which the terms are evaluated. Forward defl ation refers to the

cha9792x_ch07_176-203.indd Page 181 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 181 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

182 ROOTS OF POLYNOMIALS

case where new polynomial coeffi cients are in order of descending powers of x (that is, from the highest-order to the zero-order term). For this case, it is preferable to divide by the roots of smallest absolute value fi rst. Conversely, for backward defl ation (that is, from the zero-order to the highest-order term), it is preferable to divide by the roots of largest absolute value fi rst. Another way to reduce round-off errors is to consider each successive root estimate obtained during defl ation as a good fi rst guess. These can then be used as a starting guess, and the root determined again with the original nondefl ated polynomial. This is referred to as root polishing. Finally, a problem arises when two defl ated roots are inaccurate enough that they both converge on the same undefl ated root. In that case, you might be erroneously led to believe that the polynomial has a multiple root (recall Sec. 6.5). One way to detect this problem is to compare each polished root with those that were located previously. Press et al. (2007) discuss this problem in more detail.

7.3 CONVENTIONAL METHODS

Now that we have covered some background material on polynomials, we can begin to describe methods to locate their roots. The obvious fi rst step would be to investigate the viability of the bracketing and open approaches described in Chaps. 5 and 6. The effi cacy of these approaches depends on whether the problem being solved involves complex roots. If only real roots exist, any of the previously described methods could have utility. However, the problem of fi nding good initial guesses complicates both the bracketing and the open methods, whereas the open methods could be susceptible to divergence.

SUB poldiv(a, n, d, m, q, r) DOFOR j 5 0, n r(j) 5 a(j) q(j) 5 0 END DO DOFOR k 5 n2m, 0, 21 q(k11) 5 r(m1k) y d(m) DOFOR j 5 m1k21, k, 21 r(j) 5 r(j)2q(k11) * b(j2k) END DO END DO DOFOR j 5 m, n r(j) 5 0 END DO n 5 n2m DOFOR i 5 0, n a(i) 5 q(i11) END DO END SUB

FIGURE 7.2 Algorithm to divide a polynomial (defi ned by its coeffi cients a) by a lower-order polynomial d.

cha9792x_ch07_176-203.indd Page 182 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 182 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.4 MÜLLER’S METHOD 183

When complex roots are possible, the bracketing methods cannot be used because of the obvious problem that the criterion for defi ning a bracket (that is, sign change) does not translate to complex guesses. Of the open methods, the conventional Newton-Raphson method would provide a viable approach. In particular, concise code including defl ation can be developed. If a language that accommodates complex variables (like Fortran) is used, such an algorithm will locate both real and complex roots. However, as might be expected, it would be susceptible to convergence problems. For this reason, special methods have been devel- oped to fi nd the real and complex roots of polynomials. We describe two—the Müller and Bairstow methods—in the following sections. As you will see, both are related to the more conventional open approaches described in Chap. 6.

7.4 MÜLLER’S METHOD

Recall that the secant method obtains a root estimate by projecting a straight line to the x axis through two function values (Fig. 7.3a). Müller’s method takes a similar approach, but projects a parabola through three points (Fig. 7.3b). The method consists of deriving the coeffi cients of the parabola that goes through the three points. These coeffi cients can then be substituted into the quadratic formula to obtain the point where the parabola intercepts the x axis—that is, the root estimate. The approach is facilitated by writing the parabolic equation in a convenient form,

f2(x) 5 a(x 2 x2) 2 1 b(x 2 x2) 1 c (7.17)

We want this parabola to intersect the three points [x0, f(x0)], [x1, f(x1)], and [x2, f(x2)]. The coeffi cients of Eq. (7.17) can be evaluated by substituting each of the three points to give

f(x0) 5 a(x0 2 x2) 2 1 b(x0 2 x2) 1 c (7.18)

f(x1) 5 a(x1 2 x2) 2 1 b(x1 2 x2) 1 c (7.19)

f(x2) 5 a(x2 2 x2) 2 1 b(x2 2 x2) 1 c (7.20)

FIGURE 7.3 A comparison of two related approaches for locating roots: (a) the secant method and (b) Müller’s method.

f (x)

xx1 x0

(a)

Straight line

Root estimate

Root

f (x)

xx2 x0

(b)

Parabola

Root Root estimate

x1

cha9792x_ch07_176-203.indd Page 183 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 183 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

184 ROOTS OF POLYNOMIALS

Note that we have dropped the subscript “2” from the function for conciseness. Because we have three equations, we can solve for the three unknown coeffi cients, a, b, and c. Because two of the terms in Eq. (7.20) are zero, it can be immediately solved for c 5 f(x2). Thus, the coeffi cient c is merely equal to the function value evaluated at the third guess, x2. This result can then be substituted into Eqs. (7.18) and (7.19) to yield two equations with two unknowns:

f(x0) 2 f(x2) 5 a(x0 2 x2) 2 1 b(x0 2 x2) (7.21)

f(x1) 2 f(x2) 5 a(x1 2 x2) 2 1 b(x1 2 x2) (7.22)

Algebraic manipulation can then be used to solve for the remaining coeffi cients, a and b. One way to do this involves defi ning a number of differences,

h0 5 x1 2 x0  h1 5 x2 2 x1

d0 5 f(x1) 2 f(x0)

x1 2 x0  d1 5

f(x2) 2 f(x1) x2 2 x1

(7.23)

These can be substituted into Eqs. (7.21) and (7.22) to give

(h0 1 h1)b 2 (h0 1 h1) 2a 5 h0d0 1 h1d1

h1 b 2 h 2 1 a 5 h1d1

which can be solved for a and b. The results can be summarized as

a 5 d1 2 d0

h1 1 h0 (7.24)

b 5 ah1 1 d1 (7.25)

c 5 f(x2) (7.26)

To fi nd the root, we apply the quadratic formula to Eq. (7.17). However, because of potential round-off error, rather than using the conventional form, we use the alternative formulation [Eq. (3.13)] to yield

x3 2 x2 5 22c

b 6 2b2 2 4ac (7.27a)

or isolating the unknown x3 on the left side of the equal sign,

x3 5 x2 1 22c

b 6 2b2 2 4ac (7.27b)

Note that the use of the quadratic formula means that both real and complex roots can be located. This is a major benefi t of the method. In addition, Eq. (7.27a) provides a neat means to determine the approximate error. Because the left side represents the difference between the present (x3) and the previous (x2) root estimate, the error can be calculated as

ea 5 ` x3 2 x2x3 ` 100%

cha9792x_ch07_176-203.indd Page 184 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 184 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.4 MÜLLER’S METHOD 185

Now, a problem with Eq. (7.27a) is that it yields two roots, corresponding to the 6 term in the denominator. In Müller’s method, the sign is chosen to agree with the sign of b. This choice will result in the largest denominator, and hence, will give the root estimate that is closest to x2. Once x3 is determined, the process is repeated. This brings up the issue of which point is discarded. Two general strategies are typically used:

1. If only real roots are being located, we choose the two original points that are near- est the new root estimate, x3.

2. If both real and complex roots are being evaluated, a sequential approach is employed. That is, just like the secant method, x1, x2, and x3 take the place of x0, x1, and x2.

EXAMPLE 7.2 Müller’s Method

Problem Statement. Use Müller’s method with guesses of x0, x1, and x2 5 4.5, 5.5, and 5, respectively, to determine a root of the equation

f(x) 5 x3 2 13x 2 12

Note that the roots of this equation are 23, 21, and 4.

Solution. First, we evaluate the function at the guesses

f(4.5) 5 20.625  f(5.5) 5 82.875  f(5) 5 48

which can be used to calculate

h0 5 5.5 2 4.5 5 1  h1 5 5 2 5.5 5 20.5

d0 5 82.875 2 20.625

5.5 2 4.5 5 62.25  d1 5

48 2 82.875

5 2 5.5 5 69.75

These values in turn can be substituted into Eqs. (7.24) through (7.26) to compute

a 5 69.75 2 62.25

20.5 1 1 5 15  b 5 15(20.5) 1 69.75 5 62.25  c 5 48

The square root of the discriminant can be evaluated as

262.252 2 4(15)48 5 31.54461

Then, because Z62.25 1 31.54451Z . Z62.25 2 31.54451Z, a positive sign is employed in the denominator of Eq. (7.27b), and the new root estimate can be determined as

x3 5 5 1 22(48)

62.25 1 31.54451 5 3.976487

and develop the error estimate

ea 5 `21.0235133.976487 ` 100% 5 25.74% Because the error is large, new guesses are assigned; x0 is replaced by x1, x1 is replaced by x2, and x2 is replaced by x3. Therefore, for the new iteration,

x0 5 5.5  x1 5 5  x2 5 3.976487

cha9792x_ch07_176-203.indd Page 185 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 185 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

186 ROOTS OF POLYNOMIALS

Pseudocode to implement Müller’s method for real roots is presented in Fig. 7.4. Notice that this routine is set up to take a single initial nonzero guess that is then perturbed to develop the other two guesses. Of course, the algorithm can also be

and the calculation is repeated. The results, tabulated below, show that the method con- verges rapidly on the root, xr 5 4:

i xr Ea (%)

0 5 1 3.976487 25.74 2 4.00105 0.6139 3 4 0.0262 4 4 0.0000119

FIGURE 7.4 Pseudocode for Müller’s method.

SUB Muller(xr, h, eps, maxit) x2 5 xr x1 5 xr 1 h*xr x0 5 xr 2 h*xr DO iter 5 iter 1 1 h0 5 x1 2 x0 h1 5 x2 2 x1 d0 5 (f(x1) 2 f(x0)) / h0 d1 5 (f(x2) 2 f(x1)) / h1 a 5 (d1 2 d0) / (h1 1 h0) b 5 a*h1 1 d1 c 5 f(x2) rad 5 SQRT(b*b 2 4*a*c) If |b1rad| . |b2rad| THEN den 5 b 1 rad ELSE den 5 b 2 rad END IF dxr 5 22*c y den xr 5 x2 1 dxr PRINT iter, xr IF (|dxr| , eps*xr OR iter . 5 maxit) EXIT x0 5 x1 x1 5 x2 x2 5 xr END DO END Müller

cha9792x_ch07_176-203.indd Page 186 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 186 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.5 BAIRSTOW’S METHOD 187

programmed to accommodate three guesses. For languages like Fortran, the code will fi nd complex roots if the proper variables are declared as complex.

7.5 BAIRSTOW’S METHOD

Bairstow’s method is an iterative approach related loosely to both the Müller and Newton- Raphson methods. Before launching into a mathematical description of the technique, recall the factored form of the polynomial,

f5(x) 5 (x 1 1)(x 2 4)(x 2 5)(x 1 3)(x 2 2) (7.28)

If we divided by a factor that is not a root (for example, x 1 6), the quotient would be a fourth-order polynomial. However, for this case, a remainder would result. On the basis of the above, we can elaborate on an algorithm for determining a root of a polynomial: (1) guess a value for the root x 5 t, (2) divide the polynomial by the factor x 2 t, and (3) determine whether there is a remainder. If not, the guess was perfect and the root is equal to t. If there is a remainder, the guess can be systematically adjusted and the procedure repeated until the remainder disappears and a root is located. After this is accomplished, the entire procedure can be repeated for the quotient to locate another root. Bairstow’s method is generally based on this approach. Consequently, it hinges on the mathematical process of dividing a polynomial by a factor. Recall from our discus- sion of polynomial defl ation (Sec. 7.2.2) that synthetic division involves dividing a poly- nomial by a factor x 2 t. For example, the general polynomial [Eq. (7.1)]

fn(x) 5 a0 1 a1x 1 a2x 2 1 p 1 anxn (7.29)

can be divided by the factor x 2 t to yield a second polynomial that is one order lower,

fn21(x) 5 b1 1 b2x 1 b3x 2 1 p 1 bnxn21 (7.30)

with a remainder R 5 b0, where the coeffi cients can be calculated by the recurrence relationship

bn 5 an bi 5 ai 1 bi11t  for i 5 n 2 1 to 0

Note that if t were a root of the original polynomial, the remainder b0 would equal zero. To permit the evaluation of complex roots, Bairstow’s method divides the polynomial by a quadratic factor x2 2 rx 2 s. If this is done to Eq. (7.29), the result is a new poly- nomial

fn22(x) 5 b2 1 b3x 1 p 1 bn21 xn23 1 bnxn22

with a remainder

R 5 b1(x 2 r) 1 b0 (7.31)

As with normal synthetic division, a simple recurrence relationship can be used to perform the division by the quadratic factor:

bn 5 an (7.32a)

bn21 5 an21 1 rbn (7.32b)

bi 5 ai 1 rbi11 1 sbi12  for i 5 n 2 2 to 0 (7.32c)

cha9792x_ch07_176-203.indd Page 187 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 187 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

188 ROOTS OF POLYNOMIALS

The quadratic factor is introduced to allow the determination of complex roots. This relates to the fact that, if the coeffi cients of the original polynomial are real, the complex roots occur in conjugate pairs. If x2 2 rx 2 s is an exact divisor of the polynomial, complex roots can be determined by the quadratic formula. Thus, the method reduces to determining the values of r and s that make the quadratic factor an exact divisor. In other words, we seek the values that make the remainder term equal to zero. Inspection of Eq. (7.31) leads us to conclude that for the remainder to be zero, b0 and b1 must be zero. Because it is unlikely that our initial guesses at the values of r and s will lead to this result, we must determine a systematic way to modify our guesses so that b0 and b1 approach zero. To do this, Bairstow’s method uses a strategy similar to the Newton-Raphson approach. Because both b0 and b1 are functions of both r and s, they can be expanded using a Taylor series, as in [recall Eq. (4.26)]

b1(r 1 ¢r, s 1 ¢s) 5 b1 1 0b1 0r

¢r 1 0b1 0s

¢s

b0(r 1 ¢r, s 1 ¢s) 5 b0 1 0b0 0r

¢r 1 0b0 0s

¢s (7.33)

where the values on the right-hand side are all evaluated at r and s. Notice that second- and higher-order terms have been neglected. This represents an implicit assumption that 2r and 2s are small enough that the higher-order terms are negligible. Another way of expressing this assumption is to say that the initial guesses are adequately close to the values of r and s at the roots. The changes, Dr and Ds, needed to improve our guesses can be estimated by setting Eq. (7.33) equal to zero to give

0b1 0r

¢r 1 0b1 0s

¢s 5 2b1 (7.34)

0b0 0r

¢r 1 0b0 0s

¢s 5 2b0 (7.35)

If the partial derivatives of the b’s can be determined, these are a system of two equa- tions that can be solved simultaneously for the two unknowns, Dr and Ds. Bairstow showed that the partial derivatives can be obtained by a synthetic division of the b’s in a fashion similar to the way in which the b’s themselves were derived:

cn 5 bn (7.36a)

cn21 5 bn21 1 r cn (7.36b)

ci 5 bi 1 r ci11 1 sci12  for i 5 n 2 2 to 1 (7.36c)

where 0b0y0r 5 c1, 0b0y0s 5 0b1y0r 5 c2, and 0b1y0s 5 c3. Thus, the partial derivatives are obtained by synthetic division of the b’s. Then the partial derivatives can be substi- tuted into Eqs. (7.34) and (7.35) along with the b’s to give

c2 ¢r 1 c3 ¢s 5 2b1 c1 ¢r 1 c2 ¢s 5 2b0

cha9792x_ch07_176-203.indd Page 188 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 188 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.5 BAIRSTOW’S METHOD 189

These equations can be solved for Dr and Ds, which can in turn be employed to improve the initial guesses of r and s. At each step, an approximate error in r and s can be esti- mated, as in

Zea,r Z 5 ` ¢rr ` 100% (7.37) and

Zea,s Z 5 ` ¢ss ` 100% (7.38) When both of these error estimates fall below a prespecifi ed stopping criterion es, the values of the roots can be determined by

x 5 r 6 2r2 1 4s

2 (7.39)

At this point, three possibilities exist:

1. The quotient is a third-order polynomial or greater. For this case, Bairstow’s method would be applied to the quotient to evaluate new values for r and s. The previous values of r and s can serve as the starting guesses for this application.

2. The quotient is a quadratic. For this case, the remaining two roots could be evaluated directly with Eq. (7.39).

3. The quotient is a fi rst-order polynomial. For this case, the remaining single root can be evaluated simply as

x 5 2 s r (7.40)

EXAMPLE 7.3 Bairstow’s Method

Problem Statement. Employ Bairstow’s method to determine the roots of the polynomial

f5(x) 5 x 5 2 3.5x4 1 2.75x3 1 2.125x2 2 3.875x 1 1.25

Use initial guesses of r 5 s 5 21 and iterate to a level of es 5 1%.

Solution. Equations (7.32) and (7.36) can be applied to compute

b5 5 1  b4 5 24.5  b3 5 6.25  b2 5 0.375  b1 5 210.5 b0 5 11.375 c5 5 1  c4 5 25.5  c3 5 10.75  c2 5 24.875  c1 5 216.375

Thus, the simultaneous equations to solve for Dr and Ds are

24.875¢r 1 10.75¢s 5 10.5 216.375¢r 2 4.875¢s 5 211.375

which can be solved for Dr 5 0.3558 and Ds 5 1.1381. Therefore, our original guesses can be corrected to

r 5 21 1 0.3558 5 20.6442 s 5 21 1 1.1381 5 0.1381

cha9792x_ch07_176-203.indd Page 189 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 189 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

190 ROOTS OF POLYNOMIALS

and the approximate errors can be evaluated by Eqs. (7.37) and (7.38),

0 ea,r 0 5 ` 0.3558 20.6442

` 100% 5 55.23%   0 ea, s 0 5 ` 1.13810.1381 ` 100% 5 824.1% Next, the computation is repeated using the revised values for r and s. Applying Eqs. (7.32) and (7.36) yields

b5 5 1 b4 5 24.1442 b3 5 5.5578 b2 5 22.0276 b1 5 21.8013

b0 5 2.1304

c5 5 1 c4 5 24.7884 c3 5 8.7806 c2 5 28.3454 c1 5 4.7874

Therefore, we must solve

28.3454¢r 1 8.7806¢s 5 1.8013 4.7874¢r 2 8.3454¢s 5 22.1304

for Dr 5 0.1331 and Ds 5 0.3316, which can be used to correct the root estimates as

r 5 20.6442 1 0.1331 5 20.5111  Zea,r Z 5 26.0% s 5 0.1381 1 0.3316 5 0.4697  Zea,s Z 5 70.6%

The computation can be continued, with the result that after four iterations the method converges on values of r 5 20.5 ( Zea,r Z 5 0.063%) and s 5 0.5 (Zea,s Z 5 0.040%). Equation (7.39) can then be employed to evaluate the roots as

x 5 20.5 6 2(20.5)2 1 4(0.5)

2 5 0.5,21.0

At this point, the quotient is the cubic equation

f(x) 5 x3 2 4x2 1 5.25x 2 2.5

Bairstow’s method can be applied to this polynomial using the results of the previous step, r 5 20.5 and s 5 0.5, as starting guesses. Five iterations yield estimates of r 5 2 and s 5 21.249, which can be used to compute

x 5 2 6 222 1 4(21.249)

2 5 1 6 0.499i

At this point, the quotient is a fi rst-order polynomial that can be directly evaluated by Eq. (7.40) to determine the fi fth root: 2.

Note that the heart of Bairstow’s method is the evaluation of the b’s and c’s via Eqs. (7.32) and (7.36). One of the primary strengths of the method is the concise way in which these recurrence relationships can be programmed. Figure 7.5 lists pseudocode to implement Bairstow’s method. The heart of the algo- rithm consists of the loop to evaluate the b’s and c’s. Also notice that the code to solve the simultaneous equations checks to prevent division by zero. If this is the case, the values of r and s are perturbed slightly and the procedure is begun again. In addition, the algorithm places a user-defi ned upper limit on the number of iterations (MAXIT) and should be designed to avoid division by zero while calculating the error estimates. Finally, the algorithm requires initial guesses for r and s (rr and ss in the code). If no prior knowledge of the roots exist, they can be set to zero in the calling program.

cha9792x_ch07_176-203.indd Page 190 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 190 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.5 BAIRSTOW’S METHOD 191

(a) Bairstow Algorithm

SUB Bairstow (a,nn,es,rr,ss,maxit,re,im,ier) DIMENSION b(nn), c(nn) r 5 rr; s 5 ss; n 5 nn ier 5 0; ea1 5 1; ea2 5 1 DO IF n , 3 OR iter $ maxit EXIT iter 5 0 DO iter 5 iter 1 1 b(n) 5 a(n) b(n 2 1) 5 a(n 2 1) 1 r * b(n) c(n) 5 b(n) c(n 2 1) 5 b(n 2 1) 1 r * c(n) DO i 5 n 2 2, 0, 21 b(i) 5 a(i) 1 r * b(i 1 1) 1 s * b(i 1 2) c(i) 5 b(i) 1 r * c(i 1 1) 1 s * c(i 1 2) END DO det 5 c(2) * c(2) 2 c(3) * c(1) IF det fi 0 THEN dr 5 (2b(1) * c(2) 1 b(0) * c(3))ydet ds 5 (2b(0) * c(2) 1 b(1) * c(1))ydet r 5 r 1 dr s 5 s 1 ds IF rfi0 THEN ea1 5 ABS(dryr) * 100 IF sfi0 THEN ea2 5 ABS(dsys) * 100 ELSE r 5 r 1 1 s 5 s 1 1 iter 5 0 END IF IF ea1 # es AND ea2 # es OR iter $ maxit EXIT END DO CALL Quadroot(r,s,r1,i1,r2,i2) re(n) 5 r1 im(n) 5 i1 re(n 2 1) 5 r2 im(n 2 1) 5 i2 n 5 n 2 2 DO i 5 0, n a(i) 5 b(i 1 2) END DO END DO

IF iter , maxit THEN IF n 5 2 THEN r 5 2a(1)ya(2) s 5 2a(0)ya(2) CALL Quadroot(r,s,r1,i1,r2,i2) re(n) 5 r1 im(n) 5 i1 re(n 2 1) 5 r2 im(n 2 1) 5 i2 ELSE re(n) 5 2a(0)ya(1) im(n) 5 0 END IF ELSE ier 5 1 END IF END Bairstow

(b) Roots of Quadratic Algorithm

SUB Quadroot(r,s,r1,i1,r2,i2) disc 5 r ̂ 2 1 4 * s IF disc . 0 THEN r1 5 (r 1 SQRT(disc))y2 r2 5 (r 2 SQRT(disc))y2 i1 5 0 i2 5 0 ELSE r1 5 ry2 r2 5 r1 i1 5 SQRT(ABS(disc))y2 i2 5 2i1 END IF END QuadRoot

FIGURE 7.5 (a) Algorithm for implementing Bairstow’s method, along with (b) an algorithm to determine the roots of a quadratic.

cha9792x_ch07_176-203.indd Page 191 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 191 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

192 ROOTS OF POLYNOMIALS S O F T W A R E

7.6 OTHER METHODS

Other methods are available to locate the roots of polynomials. The Jenkins-Traub method is commonly used in software libraries. It is fairly complicated, and a good starting point to understanding it is found in Ralston and Rabinowitz (1978). Laguerre’s method, which approximates both real and complex roots and has cubic convergence, is among the best approaches. A complete discussion can be found in Householder (1970). In addition, Press et al. (2007) present a nice algorithm to imple- ment the method.

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES

Software packages have great capabilities for locating roots. In this section, we will give you a taste of some of the more useful ones.

7.7.1 Excel

A spreadsheet like Excel can be used to locate a root by trial and error. For example, if you want to fi nd a root of

f(x) 5 x 2 cos x

fi rst, you can enter a value for x in a cell. Then set up another cell for f(x) that would obtain its value for x from the fi rst cell. You can then vary the x cell until the f(x) cell approaches zero. This process can be further enhanced by using Excel’s plotting capa- bilities to obtain a good initial guess (Fig. 7.6). Although Excel does facilitate a trial-and-error approach, it also has two standard tools that can be employed for root location: Goal Seek and Solver. Both these tools can be employed to systematically adjust the initial guesses. Goal Seek is expressly used to drive an equation to a value (in our case, zero) by varying a single parameter.

FIGURE 7.6 A spreadsheet set up to determine the root of f(x) 5 x 2 cos x by trial and error. The plot is used to obtain a good initial guess.

cha9792x_ch07_176-203.indd Page 192 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 192 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES 193

EXAMPLE 7.4 Using Excel’s Goal Seek Tool to Locate a Single Root

Problem Statement. Employ Goal Seek to determine the root of the transcendental function

f(x) 5 x 2 cos x

Solution. As in Fig. 7.6, the key to solving a single equation with Excel is creating a cell to hold the value of the function in question and then making the value dependent on another cell. Once this is done, the selection Goal Seek is chosen from the What-If Analysis button on your Data ribbon. At this point a dialogue box will be displayed, asking you to set a cell to a value by changing another cell. For the example, suppose that as in Fig. 7.6 your guess is entered in cell A11 and your function result in cell B11. The Goal Seek dialogue box would be fi lled out as

When the OK button is selected, a message box displays the results,

The cells on the spreadsheet would also be modifi ed to the new values (as shown in Fig. 7.6).

The Solver tool is more sophisticated than Goal Seek in that (1) it can vary several cells simultaneously and (2) along with driving a target cell to a value, it can minimize and maximize its value. The next example illustrates how it can be used to solve a system of nonlinear equations.

EXAMPLE 7.5 Using Excel’s Solver for a Nonlinear System

Problem Statement. Recall that in Sec. 6.6 we obtained the solution of the following set of simultaneous equations,

u(x, y) 5 x2 1 xy 2 10 5 0

y(x, y) 5 y 1 3xy2 2 57 5 0

cha9792x_ch07_176-203.indd Page 193 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 193 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

194 ROOTS OF POLYNOMIALS S O F T W A R E Note that a correct pair of roots is x 5 2 and y 5 3. Use Solver to determine the roots using initial guesses of x 5 1 and y 5 3.5.

Solution. As shown below, two cells (B1 and B2) can be created to hold the guesses for x and y. The function values themselves, u(x, y) and y(x, y) can then be entered into two other cells (B3 and B4). As can be seen, the initial guesses result in function values that are far from zero.

Next, another cell can be created that contains a single value refl ecting how close both functions are to zero. One way to do this is to sum the squares of the function values. This is done and the result entered in cell B6. If both functions are at zero, this function should also be at zero. Further, using the squared functions avoids the possibility that both func- tions could have the same nonzero value, but with opposite signs. For this case, the target cell (B6) would be zero, but the roots would be incorrect. Once the spreadsheet is created, the selection Solver is chosen from the Data ribbon.1 At this point a dialogue box will be displayed, querying you for pertinent information. The pertinent cells of the Solver dialogue box would be fi lled out as

1Note that you may have to install Solver by choosing Offi ce, Excel Options, Add-Ins. Select Excel Add-Ins from the Manage drop-down box at the bottom of the Excel options menu and click Go. Then, check the Solver box. The Solver then should be installed and a button to access it should appear on your Data ribbon.

cha9792x_ch07_176-203.indd Page 194 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 194 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES 195

When the OK button is selected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the correct solution:

It should be noted that the Solver can fail. Its success depends on (1) the condition of the system of equations and/or (2) the quality of the initial guesses. Thus, the suc- cessful outcome of the previous example is not guaranteed. Despite this, we have found Solver useful enough to make it a feasible option for quickly obtaining roots in a wide range of engineering applications.

7.7.2 MATLAB

As summarized in Table 7.1, MATLAB software is capable of locating roots of single algebraic and transcendental equations. It is superb at manipulating and locating the roots of polynomials. The fzero function is designed to locate one root of a single function. A simplifi ed representation of its syntax is

fzero(f,x0,options)

where f is the function you are analyzing, x0 is the initial guess, and options are the optimization parameters (these are changed using the function optimset). If options are omitted, default values are employed. Note that one or two guesses can be employed. If two guesses are employed, they are assumed to bracket a root. The following example illustrates how fzero can be used.

TABLE 7.1 Common functions in MATLAB related to root location and polynomial manipulation.

Function Description

fzero Root of single function. roots Find polynomial roots. poly Construct polynomial with specifi ed roots. polyval Evaluate polynomial. polyvalm Evaluate polynomial with matrix argument. residue Partial-fraction expansion (residues). polyder Differentiate polynomial. conv Multiply polynomials. deconv Divide polynomials.

cha9792x_ch07_176-203.indd Page 195 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 195 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

196 ROOTS OF POLYNOMIALS S O F T W A R E EXAMPLE 7.6 Using MATLAB for Root Location

Problem Statement. Use the MATLAB function fzero to fi nd the roots of

f(x) 5 x10 2 1

within the interval xl 5 0 and xu 5 4. Obviously two roots occur at 21 and 1. Recall that in Example 5.6, we used the false-position method with initial guesses of 0 and 1.3 to determine the positive root.

Solution. Using the same initial conditions as in Example 5.6, we can use MATLAB to determine the positive root as in

>> x0=[0 1.3]; >> x=fzero(@(x) x^10–1,x0)

x = 1

In a similar fashion, we can use initial guesses of 21.3 and 0 to determine the negative root,

>> x0=[21.3 0]; >> x=fzero(@(x) x^10–1,x0)

x = –1

We can also employ a single guess. An interesting case would be to use an initial guess of 0,

>> x0=0; >> x=fzero(@(x) x^10–1,x0)

x = –1

Thus, for this guess, the underlying algorithm happens to home in on the negative root. The use of optimset can be illustrated by using it to display the actual iterations as the solution progresses:

>> x0=0; >> option=optimset('DISP','ITER'); >> x=fzero(@(x) x^10–1,x0,option)

Func–count x f(x) Procedure 1 0 –1 initial 2 –0.0282843 –1 search 3 0.0282843 –1 search 4 –0.04 –1 search • • • 21 0.64 –0.988471 search 22 –0.905097 –0.631065 search

cha9792x_ch07_176-203.indd Page 196 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 196 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES 197

23 0.905097 –0.631065 search 24 –1.28 10.8059 search

Looking for a zero in the interval [–1.28, 0.9051]

25 0.784528 –0.911674 interpolation 26 –0.247736 –0.999999 bisection 27 –0.763868 –0.932363 bisection 28 –1.02193 0.242305 bisection 29 –0.968701 –0.27239 interpolation 30 –0.996873 –0.0308299 interpolation 31 –0.999702 –0.00297526 interpolation 32 –1 5.53132e–006 interpolation 33 –1 –7.41965e–009 interpolation 34 –1 –1.88738e–014 interpolation 35 –1 0 interpolation Zero found in the interval: [–1.28, 0.9051].

x 5 21

These results illustrate the strategy used by fzero when it is provided with a single guess. First, it searches in the vicinity of the guess until it detects a sign change. Then it uses a combination of bisection and interpolation to home in on the root. The interpolation involves both the secant method and inverse quadratic interpolation (recall Sec. 7.4). It should be noted that the fzero algorithm has more to it than this basic description might imply. You can consult Press et al. (2007) for additional details.

EXAMPLE 7.7 Using MATLAB to Manipulate and Determine the Roots of Polynomials

Problem Statement. Explore how MATLAB can be employed to manipulate and de- termine the roots of polynomials. Use the following equation from Example 7.3,

f5(x) 5 x 5 2 3.5x4 1 2.75x3 1 2.125x2 2 3.875x 1 1.25 (E7.7.1)

which has three real roots: 0.5, 21.0, and 2, and one pair of complex roots: 1 6 0.5i.

Solution. Polynomials are entered into MATLAB by storing the coeffi cients as a vector. For example, at the MATLAB prompt (..) typing and entering the follow line stores the coeffi cients in the vector a,

>> a=[1 –3.5 2.75 2.125 –3.875 1.25];

We can then proceed to manipulate the polynomial. For example, we can evaluate it at x 5 1 by typing

>> polyval(a,1)

with the result 1(1)5 2 3.5(1)4 1 2.75(1)3 1 2.125(1)2 2 3.875(1) 1 1.25 5 20.25,

ans = –0.2500

cha9792x_ch07_176-203.indd Page 197 09/12/13 9:10 AM F-468 cha9792x_ch07_176-203.indd Page 197 09/12/13 9:10 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

198 ROOTS OF POLYNOMIALS S O F T W A R E We can evaluate the derivative f9(x) 5 5x

4 2 14x3 1 8.25x2 1 4.25x 2 3.875 by

>> polyder(a) ans = 5.0000 –14.0000 8.2500 4.2500 –3.8750

Next, let us create a quadratic polynomial that has roots corresponding to two of the original roots of Eq. (E7.7.1): 0.5 and 21. This quadratic is (x 2 0.5)(x 1 1) 5 x2 1 0.5x 2 0.5 and can be entered into MATLAB as the vector b,

>> b=[1 0.5 –0.5];

We can divide this polynomial into the original polynomial by

>> [d,e]5deconv(a,b)

with the result being a quotient (a third-order polynomial d) and a remainder (e),

d = 1.0000 –4.0000 5.2500 –2.5000 e = 0 0 0 0 0 0

Because the polynomial is a perfect divisor, the remainder polynomial has zero coeffi - cients. Now, the roots of the quotient polynomial can be determined as

>> roots(d)

with the expected result that the remaining roots of the original polynomial (E7.7.1) are found,

ans = 2.0000 1.0000 + 0.5000i 1.0000 – 0.5000i

We can now multiply d by b to come up with the original polynomial,

>> conv(d,b) ans = 1.0000 –3.5000 2.7500 2.1250 –3.8750 1.2500

Finally, we can determine all the roots of the original polynomial by

>> r5roots(a) r = –1.0000 2.0000 1.0000 + 0.5000i 1.0000 – 0.5000i 0.5000

cha9792x_ch07_176-203.indd Page 198 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 198 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES 199

7.7.3 Mathcad

Mathcad has a numeric mode function called root that can be used to solve an equation of a single variable. The method requires that you supply a function f(x) and either an initial guess or a bracket. When a single guess value is used, root uses the Secant and Müller methods. In the case where two guesses that bracket a root are supplied, it uses a combination of the Ridder method (a variation of false position) and Brent’s method. It iterates until the magnitude of f(x) at the proposed root is less than the predefi ned value of TOL. The Mathcad imple- mentation has similar advantages and disadvantages as conventional root location methods such as issues concerning the quality of the initial guess and the rate of convergence. Mathcad can fi nd all the real or complex roots of polynomials with polyroots. This nu- meric or symbolic mode function is based on the Laguerre method. This function does not require initial guesses, and all the roots are returned at the same time. Mathcad contains a numeric mode function called Find that can be used to solve up to 50 simultaneous nonlinear algebraic equations. The Find function chooses an appropriate method from a group of available methods, depending on whether the problem is linear or nonlinear, and other attributes. Acceptable values for the solution may be unconstrained or constrained to fall within specifi ed limits. If Find fails to locate a solution that satisfi es the equations and constraints, it returns the error message “did not fi nd solution.” However, Mathcad also contains a similar function called Minerr. This function gives solution results that mini- mize the errors in the constraints even when exact solutions cannot be found. Thus, the prob- lem of solving for the roots of nonlinear equations is closely related to both optimization and nonlinear least squares. These areas and Minerr are covered in detail in Parts Four and Five. Figure 7.7 shows a typical Mathcad worksheet. The menus at the top provide quick access to common arithmetic operators and functions, various two- and three-dimensional

FIGURE 7.7 Mathcad screen to fi nd the root of a single equation.

cha9792x_ch07_176-203.indd Page 199 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 199 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

200 ROOTS OF POLYNOMIALS S O F T W A R E

plot types, and the environment to create subprograms. Equations, text, data, or graphs can be placed anywhere on the screen. You can use a variety of fonts, colors, and styles to construct worksheets with almost any design and format that pleases you. Consult the summary of the Mathcad User’s manual in Appendix C or the full manual available from MathSoft. Note that in all our Mathcad examples, we have tried to fi t the entire Mathcad session onto a single screen. You should realize that the graph would have to be placed below the commands to work properly. Let’s start with an example that solves for the root of f(x) 5 x 2 cos x. The fi rst step is to enter the function. This is done by typing f(x): which is automatically converted to f(x):5 by Mathcad. The :5 is called the defi nition symbol. Next an initial guess is input in a similar manner using the defi nition symbol. Now, soln is defi ned as root(f(x), x), which invokes the secant method with a starting value of 1.0. Iteration is continued until f(x) evaluated at the proposed root is less than TOL. The value of TOL is set from the Math/Options pull down menu. Finally the value of soln is displayed using a normal equal sign (5). The number of signifi cant fi gures is set from the Format/Number pull down menu. The text labels and equation defi nitions can be placed anywhere on the screen in a number of different fonts, styles, sizes, and colors. The graph can be placed anywhere on the worksheet by clicking to the desired location. This places a red cross hair at that location. Then use the Insert/Graph/X-Y Plot pull down menu to place an empty plot on the worksheet with place-holders for the expressions to be graphed and for the ranges of the x and y axes. Simply type f(z) in the placeholder on the y axis and 210 and 10 for the z-axis range. Mathcad does all the rest to produce the graph shown in Fig. 7.7. Once the graph has been created you can use the Format/Graph/X-Y Plot pull down menu to vary the type of graph; change the color, type, and weight of the trace of the function; and add titles, labels and other features. Figure 7.8 shows how Mathcad can be used to fi nd the roots of a polynomial using the polyroots function. First, p(x) and v are input using the :5 defi nition symbol. Note that v is a vector that contains the coeffi cients of the polynomial starting with zero-order term and ending in this case with the third-order term. Next, r is defi ned (using :5) as polyroots(v), which invokes the Laguerre method. The roots contained in r are displayed as rT using a normal equal sign (5). Next, a plot is constructed in a manner similar to the above, except that now two range variables, x and j, are used to defi ne the range of the x axis and the location of the roots. The range variable for x is constructed by typing x and then “:” (which appears as :5) and then 24, and then “,” and then 23.99, and then “;” (which is transformed into .. by Mathcad), and fi nally 4. This creates a vector of values of x ranging from 24 to 4 with an increment of 0.01 for the x axis with corresponding values for p(x) on the y axis. The j range variable is used to create three values for r and p(r) that are plotted as individual small circles. Note that again, in our effort to fi t the entire Mathcad session onto a single screen, we have placed the graph above the commands. You should realize that the graph would have to be below the commands to work properly. The last example shows the solution of a system of nonlinear equations using a Mathcad Solve Block (Fig. 7.9). The process begins with using the defi nition symbol to create initial guesses for x and y. The word Given then alerts Mathcad that what follows is a system of equations. Then comes the equations and inequalities (not used here). Note that for this application Mathcad requires the use of a symbolic equal sign typed as [Ctrl]5 or , and . to separate the left and right sides of an equation. Now, the variable vec is defi ned as Find (x,y) and the value of vec is shown using an equal sign.

cha9792x_ch07_176-203.indd Page 200 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 200 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

7.7 ROOT LOCATION WITH SOFTWARE PACKAGES 201

FIGURE 7.8 Mathcad screen to solve for roots of polynomial.

FIGURE 7.9 Mathcad screen to solve a system of nonlinear equations.

cha9792x_ch07_176-203.indd Page 201 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 201 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

202 ROOTS OF POLYNOMIALS

PROBLEMS

7.1 Divide a polynomial f(x) 5 x4 2 7.5x3 1 14.5x2 1 3x 2 20 by the monomial factor x 2 2. Is x 5 2 a root? 7.2 Divide a polynomial f(x) 5 x5 2 5x4 1 x3 2 6x2 2 7x 1 10 by the monomial factor x 2 2. 7.3 Use Müller’s method to determine the positive real root of (a) f(x) 5 x3 1 x2 2 4x 2 4 (b) f(x) 5 x3 2 0.5x2 1 4x 2 2 7.4 Use Müller’s method or MATLAB to determine the real and complex roots of (a) f(x) 5 x3 2 x2 1 2x 2 2 (b) f(x) 5 2x4 1 6x2 1 8 (c) f(x) 5 x4 2 2x3 1 6x2 2 2x 1 5 7.5 Use Bairstow’s method to determine the roots of (a) f(x) 5 22 1 6.2x 2 4x2 1 0.7x3

(b) f(x) 5 9.34 2 21.97x 1 16.3x2 2 3.704x3

(c) f(x) 5 x4 2 2x3 1 6x2 2 2x 1 5 7.6 Develop a program to implement Müller’s method. Test it by duplicating Example 7.2. 7.7 Use the program developed in Prob. 7.6 to determine the real roots of Prob. 7.4a. Construct a graph (by hand or with a software package) to develop suitable starting guesses. 7.8 Develop a program to implement Bairstow’s method. Test it by duplicating Example 7.3. 7.9 Use the program developed in Prob. 7.8 to determine the roots of the equations in Prob. 7.5. 7.10 Determine the real root of x3.5 5 80 with Excel, MATLAB or Mathcad. 7.11 The velocity of a falling parachutist is given by

y 5 gm

c (1 2 e2 (cym)t)

where g 5 9.81 m/s2. For a parachutist with a drag coeffi cient c 5 15 kg/s, compute the mass m so that the velocity is y 5 35 m/s at t 5 8 s. Use Excel, MATLAB or Mathcad to determine m. 7.12 Determine the roots of the simultaneous nonlinear equations

y 5 2x2 1 x 1 0.75

y 1 5xy 5 x2

Employ initial guesses of x 5 y 5 1.2 and use the Solver tool from Excel or a software package of your choice. 7.13 Determine the roots of the simultaneous nonlinear equations

(x 2 4)2 1 (y 2 4)4 5 5

x2 1 y2 5 16

Use a graphical approach to obtain your initial guesses. Determine refi ned estimates with the Solver tool from Excel or a software package of your choice.

7.14 Perform the identical MATLAB operations as those in Example 7.7 or use a software package of your choice to fi nd all the roots of the polynomial

f(x) 5 (x 1 2)(x 1 5)(x 2 6)(x 2 4)(x 2 8)

Note that the poly function can be used to convert the roots to a polynomial. 7.15 Use MATLAB or Mathcad to determine the roots for the equations in Prob. 7.5. 7.16 A two-dimensional circular cylinder is placed in a high-speed uniform fl ow. Vortices shed from the cylinder at a constant frequency, and pressure sensors on the rear surface of the cylinder detect this frequency by calculating how often the pressure oscil- lates. Given three data points, use Müller’s method to fi nd the time where the pressure was zero.

Time 0.60 0.62 0.64

Pressure 20 50 60

7.17 When trying to find the acidity of a solution of magne- sium hydroxide in hydrochloric acid, we obtain the following equation

A(x) 5 x3 1 3.5x2 2 40

where x is the hydronium ion concentration. Find the hydronium ion concentration for a saturated solution (acidity equals zero) using two different methods in MATLAB (for example, graphically and the roots function). 7.18 Consider the following system with three unknowns a, u, and y:

u2 2 2y2 5 a2

u 1 y 5 2

a2 2 2a 2 u 5 0

Solve for the real values of the unknowns using: (a) the Excel Solver and (b) a symbolic manipulator software package. 7.19 In control systems analysis, transfer functions are developed that mathematically relate the dynamics of a system’s input to its output. A transfer function for a robotic positioning system is given by

G(s) 5 C(s)

N(s) 5

s3 1 9s2 1 26s 1 24

s4 1 15s3 1 77s2 1 153s 1 90

where G(s) 5 system gain, C(s) 5 system output, N(s) 5 system input, and s 5 Laplace transform complex frequency. Use a

cha9792x_ch07_176-203.indd Page 202 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 202 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 203

numerical technique to fi nd the roots of the numerator and denomi- nator and factor these into the form

G(s) 5 (s 1 a1) (s 1 a2) (s 1 a3)

(s 1 b1) (s 1 b2) (s 1 b3) (s 1 b4)

where ai and bi 5 the roots of the numerator and denominator, respectively. 7.20 Develop an M-fi le function for bisection in a similar fashion to Fig. 5.10. Test the function by duplicating the computations from Examples 5.3 and 5.4. 7.21 Develop an M-fi le function for the false-position method. The structure of your function should be similar to the bisection algorithm outlined in Fig. 5.10. Test the program by duplicating Example 5.5.

7.22 Develop an M-fi le function for the Newton-Raphson method based on Fig. 6.4 and Sec. 6.2.3. Along with the initial guess, pass the function and its derivative as arguments. Test it by duplicating the computation from Example 6.3. 7.23 Develop an M-fi le function for the secant method based on Fig. 6.4 and Sec. 6.3.2. Along with the two initial guesses, pass the function as an argument. Test it by duplicating the computation from Example 6.6. 7.24 Develop an M-fi le function for the modifi ed secant method based on Fig. 6.4 and Sec. 6.3.2. Along with the initial guess and the perturbation fraction, pass the function as an argument. Test it by duplicating the computation from Example 6.8.

cha9792x_ch07_176-203.indd Page 203 15/10/13 7:16 PM f-479 cha9792x_ch07_176-203.indd Page 203 15/10/13 7:16 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

204

8 C H A P T E R 8

Case Studies: Roots of Equations

The purpose of this chapter is to use the numerical procedures discussed in Chaps. 5, 6, and 7 to solve actual engineering problems. Numerical techniques are important for practical applications because engineers frequently encounter problems that cannot be approached using analytical techniques. For example, simple mathematical models that can be solved analytically may not be applicable when real problems are involved. Thus, more complicated models must be employed. For these cases, it is appropriate to imple- ment a numerical solution on a computer. In other situations, engineering design prob- lems may require solutions for implicit variables in complicated equations. The following case studies are typical of those that are routinely encountered during upper-class courses and graduate studies. Furthermore, they are representative of prob- lems you will address professionally. The problems are drawn from the four major disciplines of engineering: chemical, civil, electrical, and mechanical. These applications also serve to illustrate the trade-offs among the various numerical techniques. The fi rst application, taken from chemical engineering, provides an excellent example of how root-location methods allow you to use realistic formulas in engineering practice. In addition, it also demonstrates how the effi ciency of the Newton-Raphson technique is used to advantage when a large number of root-location computations is required. The following engineering design problems are taken from civil, electrical, and mechan- ical engineering. Section 8.2 uses bisection to determine changes in rainwater chemistry due to increases in atmospheric carbon dioxide. Section 8.3 shows how the roots of transcendental equations can be used in the design of an electrical circuit. Sections 8.2 and 8.3 also illustrate how graphical methods provide insight into the root-location process. Finally, Sec. 8.4 uses a variety of numerical methods to compute the friction factor for fl uid fl ow in a pipe.

8.1 IDEAL AND NONIDEAL GAS LAWS (CHEMICAL/BIO ENGINEERING)

Background. The ideal gas law is given by

pV 5 nRT (8.1)

where p is the absolute pressure, V is the volume, n is the number of moles, R is the universal gas constant, and T is the absolute temperature. Although this equation is

cha9792x_ch08_204-225.indd Page 204 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 204 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8.1 IDEAL AND NONIDEAL GAS LAWS 205

widely used by engineers and scientists, it is accurate over only a limited range of pres- sure and temperature. Furthermore, Eq. (8.1) is more appropriate for some gases than for others. An alternative equation of state for gases is given by

ap 1 a y2 b(y 2 b) 5 RT (8.2)

known as the van der Waals equation, where y 5 V/n is the molal volume and a and b are empirical constants that depend on the particular gas. A chemical engineering design project requires that you accurately estimate the molal volume (y) of both carbon dioxide and oxygen for a number of different temperature and pressure combinations so that appropriate containment vessels can be selected. It is also of interest to examine how well each gas conforms to the ideal gas law by comparing the molal volume as calculated by Eqs. (8.1) and (8.2). The following data are provided:

R 5 0.082054 L atm / (mol K)

a 5 3.592

b 5 0.04267 f carbon dioxide

a 5 1.360

b 5 0.03183 f oxygen

The design pressures of interest are 1, 10, and 100 atm for temperature combinations of 300, 500, and 700 K.

Solution. Molal volumes for both gases are calculated using the ideal gas law, with n 5 1. For example, if p 5 1 atm and T 5 300 K,

y 5 V n

5 RT p

5 0.082054 L atm

mol K 300 K

1 atm 5 24.6162 L/mol

These calculations are repeated for all temperature and pressure combinations and presented in Table 8.1.

TABLE 8.1 Computations of molal volume.

Molal Volume Molal Volume Molal Volume (van der Waals) (van der Waals) Temperature, Pressure, (Ideal Gas Law), Carbon Dioxide, Oxygen, K atm L/mol L/mol L/mol

300 1 24.6162 24.5126 24.5928 10 2.4616 2.3545 2.4384 100 0.2462 0.0795 0.2264 500 1 41.0270 40.9821 41.0259 10 4.1027 4.0578 4.1016 100 0.4103 0.3663 0.4116 700 1 57.4378 57.4179 57.4460 10 5.7438 5.7242 5.7521 100 0.5744 0.5575 0.5842

cha9792x_ch08_204-225.indd Page 205 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 205 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

206 CASE STUDIES: ROOTS OF EQUATIONS

The computation of molal volume from the van der Waals equation can be accom- plished using any of the numerical methods for fi nding roots of equations discussed in Chaps. 5, 6, and 7, with

f(y) 5 ap 1 a y2 b(y 2 b) 2 RT (8.3)

In this case, the derivative of f(y) is easy to determine and the Newton-Raphson method is convenient and effi cient to implement. The derivative of f(y) with respect to y is given by

f ¿(y) 5 p 2 a

y2 1

2ab

y3 (8.4)

The Newton-Raphson method is described by Eq. (6.6):

yi11 5 yi 2 f(yi)

f ¿(yi)

which can be used to estimate the root. For example, using the initial guess of 24.6162, the molal volume of carbon dioxide at 300 K and 1 atm is computed as 24.5126 L/mol. This result was obtained after just two iterations and has an ea of less than 0.001 percent. Similar computations for all combinations of pressure and temperature for both gases are presented in Table 8.1. It is seen that the results for the ideal gas law differ from those for the van der Waals equation for both gases, depending on specifi c values for p and T. Furthermore, because some of these results are signifi cantly different, your design of the containment vessels would be quite different, depending on which equation of state was used. In this case, a complicated equation of state was examined using the Newton-Raphson method. The results varied signifi cantly from the ideal gas law for several cases. From a practical standpoint, the Newton-Raphson method was appropriate for this application because f9(y) was easy to calculate. Thus, the rapid convergence properties of the Newton-Raphson method could be exploited. In addition to demonstrating its power for a single computation, the present design problem also illustrates how the Newton-Raphson method is especially attractive when numerous computations are required. Because of the speed of digital computers, the effi ciency of various numerical methods for most roots of equations is indistinguishable for a single computation. Even a 1-s difference between the crude bisection approach and the effi cient Newton-Raphson does not amount to a signifi cant time loss when only one computation is performed. However, suppose that millions of root evaluations are required to solve a problem. In this case, the effi ciency of the method could be a decid- ing factor in the choice of a technique. For example, suppose that you are called upon to design an automatic computerized control system for a chemical production process. This system requires accurate estimates of molal volumes on an essentially continuous basis to properly manufacture the fi nal product. Gauges are installed that provide instantaneous readings of pressure and tempera- ture. Evaluations of y must be obtained for a variety of gases that are used in the process. For such an application, bracketing methods such as bisection or false position would probably be too time-consuming. In addition, the two initial guesses that are required for

cha9792x_ch08_204-225.indd Page 206 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 206 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8.2 GREENHOUSE GASES AND RAINWATER 207

these approaches may also interject a critical delay in the procedure. This shortcoming is relevant to the secant method, which also needs two initial estimates. In contrast, the Newton-Raphson method requires only one guess for the root. The ideal gas law could be employed to obtain this guess at the initiation of the process. Then, assuming that the time frame is short enough so that pressure and temperature do not vary wildly between computations, the previous root solution would provide a good guess for the next application. Thus, the close guess that is often a prerequisite for con- vergence of the Newton-Raphson method would automatically be available. All the above considerations would greatly favor the Newton-Raphson technique for such problems.

8.2 GREENHOUSE GASES AND RAINWATER (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. Civil engineering is a broad fi eld that includes such diverse areas as structural, geotechnical, transportation, water-resources, and environmental engineering. The last area has traditionally dealt with pollution control. However, in recent years, environmental engineers (as well as chemical engineers) have addressed broader problems such as climate change. It is well documented that the atmospheric levels of several greenhouse gases have been increasing over the past 50 years. For example, Fig. 8.1 shows data for the partial pressure of carbon dioxide (CO2) collected at Mauna Loa, Hawaii, from 1958 through 2003. The trend in the data can be nicely fi t with a quadratic polynomial (in Part Five, we will learn how to determine such polynomials),

pCO2 5 0.011825(t 2 1980.5) 2 1 1.356975(t 2 1980.5) 1 339

where pCO2 5 the partial pressure of CO2 in the atmosphere [ppm]. The data indicate that levels have increased over 19% during the period from 315 to 376 ppm.

FIGURE 8.1 Average annual partial pressures of atmospheric carbon dioxide (ppm) measured at Mauna Loa, Hawaii.

310

330

350

370

1950 1960 1970 1980 1990 2000 2010

pCO2 (ppm)

cha9792x_ch08_204-225.indd Page 207 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 207 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

208 CASE STUDIES: ROOTS OF EQUATIONS

Aside from global warming, greenhouse gases can also infl uence atmospheric chemistry. One question that we can address is how the carbon dioxide trend is affecting the pH of rainwater. Outside of urban and industrial areas, it is well documented that carbon dioxide is the primary determinant of the pH of the rain. pH is the measure of the activity of hydrogen ions and, therefore, its acidity. For dilute aqueous solutions, it can be computed as

pH 5 2log10[H 1] (8.5)

where [H1] is the molar concentration of hydrogen ions. The following fi ve nonlinear system of equations govern the chemistry of rainwater,

K1 5 10 6

[H1][HCO23 ]

KH pCO2 (8.6)

K2 5 [H1][CO223 ]

[HCO23 ] (8.7)

Kw 5 [H 1][OH2] (8.8)

cT 5 KH pCO2

106 1 [HCO23 ] 1 [CO

22 3 ] (8.9)

0 5 [HCO23 ] 1 2[CO 22 3 ] 1 [OH

2] 2 [H1] (8.10)

where KH 5 Henry’s constant, and K1, K2, and Kw are equilibrium coeffi cients. The fi ve unknowns in this system of fi ve nonlinear equations are cT 5 total inorganic carbon, [HCO23 ] 5 bicarbonate, [CO

22 3 ] 5 carbonate, [H

1] 5 hydrogen ion, and [OH2] 5 hydroxyl ion. Notice how the partial pressure of CO2 shows up in Eqs. (8.6) and (8.9). Use these equations to compute the pH of rainwater given that KH 5 10

21.46, K1 5 10

26.3, K2 5 10 210.3, and Kw 5 10

214. Compare the results in 1958 when the pCO2 was 315 and in 2003 when it was 375 ppm. When selecting a numerical method for your computation, consider the following:

You know with certainty that the pH of rain in pristine areas always falls between 2 and 12.

You also know that your measurement devices can only measure pH to two places of decimal precision.

Solution. There are a variety of ways to solve this nonlinear system of fi ve equations. One way is to eliminate unknowns by combining them to produce a single function that only depends on [H1]. To do this, fi rst solve Eqs. (8.6) and (8.7) for

[HCO23 ] 5 K1

106[H1] KH pCO2 (8.11)

[CO223 ] 5 K2[HCO

2 3 ]

[H1] (8.12)

Substitute Eq. (8.11) into (8.12)

[CO223 ] 5 K2K1

106[H1]2 KH pCO2 (8.13)

cha9792x_ch08_204-225.indd Page 208 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 208 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8.3 DESIGN OF AN ELECTRIC CIRCUIT 209

Equations (8.11) and (8.13) can be substituted along with Eq. (8.8) into Eq. (8.10) to give

0 5 K1

106[H1] KH pCO2 1 2

K2K1 106[H1]2

KH pCO2 1 Kw

[H1] 2 [H1] (8.14)

Although it might not be apparent, this result is a third-order polynomial in [H1]. Thus, its root can be used to compute the pH of the rainwater. Now we must decide which numerical method to employ to obtain the solution. There are two reasons why bisection would be a good choice. First, the fact that the pH always falls within the range from 2 to 12 provides us with two good initial guesses. Second, because the pH can only be measured to two decimal places of precision, we will be satisfi ed with an absolute error of Ea,d 5 0.005. Remember that given an initial bracket and the desired relative error, we can compute the number of iterations a priori. Using Eq. (5.5), the result is n 5 log2(10)�0.005 5 10.9658. Thus, eleven iterations of bisection will produce the desired precision. If this is done, the result for 1958 will be a pH of 5.6279 with a relative error of 0.0868%. We can be confi dent that the rounded result of 5.63 is correct to two decimal places. This can be verifi ed by performing another run with more iterations. For example, if we perform 35 iterations, a result of 5.6304 is obtained with an approximate relative error of εa 5 5.17 3 1029%. The same calculation can be repeated for the 2003 condi- tions to give pH 5 5.59 with εa 5 0.0874%. Interestingly, these results indicate that the 19% rise in atmospheric CO2 levels has produced only a 0.67% drop in pH. Although this is certainly true, remember that the pH represents a logarithmic scale as defi ned by Eq. (8.5). Consequently, a unit drop in pH represents a 10-fold increase in hydrogen ion. The concentration can be computed as [H1] 5 102pH and the resulting percent change can be calculated as 9.1%. Therefore, the hydrogen ion concentration has increased about 9%. There is quite a lot of controversy related to the true signifi cance of the greenhouse gas trends. However, regardless of the ultimate implications, it is quite sobering to realize that something as large as our atmosphere has changed so much over a relatively short time period. This case study illustrates how numerical methods can be employed to analyze and interpret such trends. Over the coming years, engineers and scientists can hopefully use such tools to gain increased understanding and help rationalize the debate over their ramifi cations.

8.3 DESIGN OF AN ELECTRIC CIRCUIT (ELECTRICAL ENGINEERING)

Background. Electrical engineers often use Kirchhoff’s laws to study the steady-state (not time-varying) behavior of electric circuits. Such steady-state behavior will be exam- ined in Sec. 12.3. Another important problem involves circuits that are transient in nature where sudden temporal changes take place. Such a situation occurs following the closing of the switch in Fig. 8.2. In this case, there will be a period of adjustment following the closing of the switch as a new steady state is reached. The length of this adjustment period is closely related to the storage properties of the capacitor and the inductor. Energy storage may oscillate between these two elements during a transient period. However, resistance in the circuit will dissipate the magnitude of the oscillations.

cha9792x_ch08_204-225.indd Page 209 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 209 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

210 CASE STUDIES: ROOTS OF EQUATIONS

The fl ow of current through the resistor causes a voltage drop (VR) given by

VR 5 iR

where i 5 the current and R 5 the resistance of the resistor. When R and i have units of ohms and amperes, respectively, VR has units of volts. Similarly, an inductor resists changes in current, such that the voltage drop VL across it is

VL 5 L di

dt

where L 5 the inductance. When L and i have units of henrys and amperes, respectively, VL has units of volts and t has units of seconds. The voltage drop across the capacitor (VC) depends on the charge (q) on it:

VC 5 q

C (8.15)

where C 5 the capacitance. When the charge is expressed in units of coulombs, the unit of C is the farad. Kirchhoff’s second law states that the algebraic sum of voltage drops around a closed circuit is zero. After the switch is closed we have

L di

dt 1 Ri 1

q

C 5 0 (8.16)

However, the current is related to the charge according to

i 5 dq

dt (8.17)

Therefore,

L d 2q

dt 2 1 R

dq

dt 1

1

C q 5 0 (8.18)

This is a second-order linear ordinary differential equation that can be solved using the methods of calculus. This solution is given by

q(t) 5 q0e 2Rty(2L)cos c

B

1

LC 2 a R

2L b2t d (8.19)

FIGURE 8.2 An electric circuit. When the switch is closed, the current will undergo a series of oscillations until a new steady state is reached.

Switch

Resistor

Capacitor –

+ V0

i –

+ Battery Inductor

cha9792x_ch08_204-225.indd Page 210 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 210 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8.3 DESIGN OF AN ELECTRIC CIRCUIT 211

where at t 5 0, q 5 q0 5 V0C, and V0 5 the voltage from the charging battery. Equation (8.19) describes the time variation of the charge on the capacitor. The solution q(t) is plotted in Fig. 8.3. A typical electrical engineering design problem might involve determining the proper resistor to dissipate energy at a specifi ed rate, with known values for L and C. For this prob- lem, assume the charge must be dissipated to 1 percent of its original value (q�q0 5 0.01) in t 5 0.05 s, with L 5 5 H and C 5 1024 F.

Solution. It is necessary to solve Eq. (8.19) for R, with known values of q, q0, L, and C. However, a numerical approximation technique must be employed because R is an implicit variable in Eq. (8.19). The bisection method will be used for this purpose. The other methods discussed in Chaps. 5 and 6 are also appropriate, although the Newton- Raphson method might be deemed inconvenient because the derivative of Eq. (8.19) is a little cumbersome. Rearranging Eq. (8.19),

f(R) 5 e2Rty(2L)cos c B

1

LC 2 a R

2L b2t d 2 q

q0

or using the numerical values given,

f(R) 5 e20.005Rcos[22000 2 0.01R2 (0.05)] 2 0.01 (8.20)

Examination of this equation suggests that a reasonable initial range for R is 0 to 400 V (because 2000 2 0.01R2 must be greater than zero). Figure 8.4, a plot of Eq. (8.20), confi rms this. Twenty-one iterations of the bisection method give R 5 328.1515 V, with an error of less than 0.0001 percent.

FIGURE 8.3 The charge on a capacitor as a function of time following the closing of the switch in Fig. 8.2.

q(t) q0

Time

FIGURE 8.4 Plot of Eq. (8.20) used to obtain initial guesses for R that bracket the root.

f (R)

R 0.0

– 0.2

– 0.4

– 0.6

200

Root � 325

400

cha9792x_ch08_204-225.indd Page 211 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 211 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

212 CASE STUDIES: ROOTS OF EQUATIONS

Thus, you can specify a resistor with this rating for the circuit shown in Fig. 8.2 and expect to achieve a dissipation performance that is consistent with the requirements of the problem. This design problem could not be solved effi ciently without using the numerical methods in Chaps. 5 and 6.

8.4 PIPE FRICTION (MECHANICAL/AEROSPACE ENGINEERING)

Background. Determining fl uid fl ow through pipes and tubes has great relevance in many areas of engineering and science. In mechanical and aerospace engineering, typical applications include the fl ow of liquids and gases through cooling systems. The resistance to fl ow in such conduits is parameterized by a dimensionless number called the friction factor. For turbulent fl ow, the Colebrook equation provides a means to calculate the friction factor,

0 5 1

1f 1 2.0 log a e

3.7D 1

2.51

Re1f b (8.21)

where � 5 the roughness (m), D 5 diameter (m), and Re 5 the Reynolds number,

Re 5 rVD m

where � 5 the fl uid’s density (kg/m3), V 5 its velocity (m/s), and � 5 dynamic viscos- ity (N ? s/m2). In addition to appearing in Eq. (8.21), the Reynolds number also serves as the criterion for whether fl ow is turbulent (Re . 4000).

In the present case study, we will illustrate how the numerical methods covered in this part of the book can be employed to determine f for air fl ow through a smooth, thin tube. For this case, the parameters are � 5 1.23 kg/m3, � 5 1.79 3 1025 N ? s/m2, D 5 0.005 m, V 5 40 m/s, and � 5 0.0015 mm. Note that friction factors range from about 0.008 to 0.08. In addition, an explicit formulation called the Swamee-Jain equation provides an approxi- mate estimate,

f 5 1.325

c ln a e 3.7D

1 5.74

Re0.9 b d 2

(8.22)

Solution. The Reynolds number can be computed as

Re 5 rVD m

5 1.23(40)0.005

1.79 3 1025 5 13,743

This value along with the other parameters can be substituted into Eq. (8.21) to give

g( f ) 5 1

1f 1 2.0 log a 0.0000015

3.7(0.005) 1

2.51

13,7431f b

Before determining the root, it is advisable to plot the function to estimate initial guesses and to anticipate possible diffi culties. This can be done easily with tools such

cha9792x_ch08_204-225.indd Page 212 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 212 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8.4 PIPE FRICTION 213

as MATLAB software, Excel, or Mathcad. For example, a plot of the function can be generated with the following MATLAB commands

>> rho=1.23;mu=1.79e-5;D=0.005;V=40;e=0.0015/1000; >> Re=rho*V*D/mu; >> g=@(f) 1/sqrt(f)+2*log10(e/(3.7*D)+2.51/(Re*sqrt(f))); >> fplot(g,[0.008 0.08]),grid,xlabel('f'),ylabel('g(f)')

As in Fig. 8.5, the root is located at about 0.03. Because we are supplied initial guesses (xl 5 0.008 and xu 5 0.08), either of the bracketing methods from Chap. 5 could be used. For example, bisection gives a value of f 5 0.0289678 with a percent relative error of error of 5.926 3 1025 in 22 iterations. False position yields a result of similar precision in 26 iterations. Thus, although they produce the correct result, they are somewhat ineffi cient. This would not be important for a single application, but could become prohibitive if many evaluations were made. We could try to attain improved performance by turning to an open method. Because Eq. (8.21) is relatively straightforward to differentiate, the Newton-Raphson method is a good candidate. For example, using an initial guess at the lower end of the range (x0 5 0.008), Newton-Raphson converges quickly to 0.0289678 with an approximate error of 6.87 3 1026% in only 6 iterations. However, when the initial guess is set at the upper end of the range (x0 5 0.08), the routine diverges! As can be seen by inspecting Fig. 8.5, this occurs because the function’s slope at the initial guess causes the fi rst iteration to jump to a negative value. Further runs demonstrate that for this case, convergence only occurs when the initial guess is below about 0.066.

FIGURE 8.5

–3 0.020.01 0.03 0.04 0.05 0.06 0.07 0.08

–2

–1

0

1

2

3

4

5

6

g (f

) f

cha9792x_ch08_204-225.indd Page 213 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 213 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

214 CASE STUDIES: ROOTS OF EQUATIONS

So we can see that although the Newton-Raphson is very effi cient, it requires good initial guesses. For the Colebrook equation, a good strategy might be to employ the Swamee-Jain equation (Eq. 8.22) to provide the initial guess as in

f 5 1.325

c ln a 0.0000015 3.7(0.005)

1 5.74

137430.9 b d 2

5 0.029031

For this case, Newton-Raphson converges in only 3 iterations quickly to 0.0289678 with an approximate error of 8.51 3 10210%. Aside from our homemade functions, we can also use professional root fi nders like MATLAB’s built-in fzero function. However, just as with the Newton-Raphson method, divergence also occurs when fzero function is used with a single guess. However, in this case, guesses at the lower end of the range cause problems. For example,

>> rho=1.23;mu=1.79e-5;D=0.005;V=40;e=0.0015/1000; >> Re=rho*V*D/mu >> g=@(f) 1/sqrt(f)+2*log10(e/(3.7*D)+2.51/(Re*sqrt(f))); >> fzero(g,0.008)

Exiting fzero: aborting search for an interval containing a sign change because complex function value encountered during search. (Function value at -0.0028 is -4.92028- 20.2423i.) Check function or try again with a different starting value. ans = NaN

If the iterations are displayed using optimset (recall Sec. 7.7.2), it is revealed that a negative value occurs during the search phase before a sign change is detected and the routine aborts. However, for single initial guesses above about 0.016, the routine works nicely. For example, for the guess of 0.08 that caused problems for Newton-Raphson, fzero does just fi ne,

>> fzero(g,0.08)

ans = 0.02896781017144

As a fi nal note, let’s see whether convergence is possible for simple fi xed-point iteration. The easiest and most straightforward version involves solving for the fi rst f in Eq. (8.21),

fi11 5 0.25

alog a e 3.7D

1 2.51

Re1fi bb2

(8.23)

The cobweb display of this function depicted indicates a surprising result (Fig. 8.6). Recall that fi xed-point iteration converges when the y2 curve has a relatively fl at slope (i.e., Zg9(�)Z , 1). As indicated by Fig. 8.6, the fact that the y2 curve is quite fl at in the range from f 5 0.008 to 0.08 means that not only does fi xed-point iteration converge,

cha9792x_ch08_204-225.indd Page 214 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 214 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 215

but it converges fairly rapidly! In fact, for initial guesses anywhere between 0.008 and 0.08, fi xed-point iteration yields predictions with percent relative errors less than 0.008% in six or fewer iterations. Thus, this simple approach that requires only one guess and no derivative estimates performs really well for this particular case. The take-home message from this case study is that even great, professionally- developed software like MATLAB is not always foolproof. Further, there is usually no single method that works best for all problems. Sophisticated users understand the strengths and weaknesses of the available numerical techniques. In addition, they under- stand enough of the underlying theory so that they can effectively deal with situations where a method breaks down.

FIGURE 8.6

0

0.01

0.02

0.03

0.04

0.05

0 0.02 0.04 0.06 0.08

y2 = g(x)

y1 = x

x

y

PROBLEMS

Chemical/Bio Engineering 8.1 Perform the same computation as in Sec. 8.1, but for ethyl alcohol (a 5 12.02 and b 5 0.08407) at a temperature of 375 K and p of 2.0 atm. Compare your results with the ideal gas law. Use any of the numerical methods discussed in Chaps. 5 and 6 to perform the computation. Justify your choice of technique. 8.2 In chemical engineering, plug fl ow reactors (that is, those in which fl uid fl ows from one end to the other with minimal mixing along the longitudinal axis) are often used to convert reactants into products. It has been determined that the effi ciency of the conver- sion can sometimes be improved by recycling a portion of the

product stream so that it returns to the entrance for an additional pass through the reactor (Fig. P8.2). The recycle rate is defi ned as

R 5 volume of fluid returned to entrance

volume leaving the system

Suppose that we are processing a chemical A to generate a product B. For the case where A forms B according to an autocatalytic reac- tion (that is, in which one of the products acts as a catalyst or stimulus for the reaction), it can be shown that an optimal recycle rate must satisfy

cha9792x_ch08_204-225.indd Page 215 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 215 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

216 CASE STUDIES: ROOTS OF EQUATIONS

moles of C that are produced. Conservation of mass can be used to reformulate the equilibrium relationship as

K 5 (cc,0 1 x)

(ca,0 2 2x) 2(cb,0 2 x)

where the subscript 0 designates the initial concentration of each constituent. If K 5 0.015, ca,0 5 42, cb,0 5 30, and cc,0 5 4, determine the value of x. (a) Obtain the solution graphically. (b) On the basis of (a), solve for the root with initial guesses of xl 5 0 and xu 5 20 to �s 5 0.5%. Choose either bisection or false position to obtain your solution. Justify your choice. 8.6 The following chemical reactions take place in a closed system

2A 1 B C

A 1 D C

At equilibrium, they can be characterized by

K1 5 cc

c2a cb

K2 5 cc

ca cd

where the nomenclature represents the concentration of constituent i. If x1 and x2 are the number of moles of C that are produced due to the fi rst and second reactions, respectively, use an approach similar to that of Prob. 8.5 to reformulate the equilibrium relationships in terms of the initial concentrations of the constituents. Then, use the Newton-Raphson method to solve the pair of simultaneous non- linear equations for x1 and x2 if K1 5 4 3 10

24, K2 5 3.7 3 10 22,

ca,0 5 50, cb,0 5 20, cc,0 5 5, and cd,0 5 10. Use a graphical approach to develop your initial guesses. 8.7 The Redlich-Kwong equation of state is given by

p 5 RT

y 2 b 2

a

y(y 1 b)1T

where R 5 the universal gas constant [5 0.518 kJ/(kg K)], T 5 absolute temperature (K), p 5 absolute pressure (kPa), and y 5 the volume of a kg of gas (m3/kg). The parameters a and b are calculated by

a 5 0.427 R2 T 2.5c

pc      b 5 0.0866R

Tc pc

where pc 5 critical pressure (kPa) and Tc 5 critical temperature (K). As a chemical engineer, you are asked to determine the amount of methane fuel (pc 5 4600 kPa and Tc 5 191 K) that can be held in a 3-m3 tank at a temperature of 2408C with a pressure of 65,000 kPa. Use a root-locating method of your choice to calculate y and then determine the mass of methane contained in the tank.

ln 1 1 R(1 2 XAf)

R(1 2 XAf) 5

R 1 1

R[1 1 R(1 2 XAf)]

where XAf 5 the fraction of reactant A that is converted to product B. The optimal recycle rate corresponds to the minimum-sized reactor needed to attain the desired level of conversion. Use a numerical method to determine the recycle ratios needed to mini- mize reactor size for a fractional conversion of XAf 5 0.9. 8.3 In a chemical engineering process, water vapor (H2O) is heated to suffi ciently high temperatures that a signifi cant portion of the water dissociates, or splits apart, to form oxygen (O2) and hydrogen (H2):

H2O H2 1 1 2 O2

If it is assumed that this is the only reaction involved, the mole fraction x of H2O that dissociates can be represented by

K 5 x

1 2 x

A

2pt 2 1 x

(P8.3.1)

where K 5 the reaction equilibrium constant and pt 5 the total pressure of the mixture. If pt 5 3 atm and K 5 0.05, determine the value of x that satisfi es Eq. (P8.3.1). 8.4 The following equation pertains to the concentration of a chemical in a completely mixed reactor:

c 5 cin(1 2 e20.04t) 1 c0e20.04t

If the initial concentration c0 5 4 and the infl ow concentration cin 5 10, compute the time required for c to be 93 percent of cin. 8.5 A reversible chemical reaction

2A 1 B C

can be characterized by the equilibrium relationship

K 5 cc

c2a cb

where the nomenclature ci represents the concentration of constituent i. Suppose that we defi ne a variable x as representing the number of

FIGURE P8.2 Schematic representation of a plug fl ow reactor with recycle.

Plug flow reactor

Recycle

Feed Product

cha9792x_ch08_204-225.indd Page 216 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 216 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 217

Given the parameter values listed below, fi nd the void fraction e of the bed.

DpGo m

5 1000

¢PrDp G2oL

5 10

8.13 The pressure drop in a section of pipe can be calculated as

¢p 5 f LrV 2

2D

where Dp 5 the pressure drop (Pa), f 5 the friction factor, L 5 the length of pipe [m], � 5 density (kg/m3), V 5 velocity (m /s), and D 5 diameter (m). For turbulent fl ow, the Colebrook equation pro- vides a means to calculate the friction factor,

1

1f 5 22.0 log a e

3.7D 1

2.51

Re1f b

where � 5 the roughness (m), and Re 5 the Reynolds number,

Re 5 rVD

m

where � 5 dynamic viscosity (N ? s/m2). (a) Determine Dp for a 0.2-m-long horizontal stretch of smooth

drawn tubing given � 5 1.23 kg/m3, m 5 1.79 3 1025 N ? s/m2, D 5 0.005 m, V 5 40 m/s, and e 5 0.0015 mm. Use a numerical method to determine the friction factor. Note that smooth pipes with Re , 105, a good initial guess can be obtained using the Blasius formula, f 5 0.316yRe0.25.

(b) Repeat the computation but for a rougher commercial steel pipe (e 5 0.045 mm).

Civil and Environmental Engineering 8.14 In structural engineering, the secant formula defi nes the force per unit area, PyA, that causes a maximum stress �m in a column of given slenderness ratio Lyk:

P

A 5

sm

1 1 (ecyk2)sec[0.51Py(EA) (Lyk)]

where ecyk2 5 the eccentricity ratio and E 5 the modulus of elasticity. If for a steel beam, E 5 200,000 MPa, ecyk2 5 0.2, and sm 5 250 MPa, compute PyA for Lyk 5 100. Recall that sec x 5 1ycos x. 8.15 In environmental engineering (a specialty area in civil engineering), the following equation can be used to compute the oxy- gen level c (mg/L) in a river downstream from a sewage discharge:

c 5 10 2 20(e20.2x 2 e20.75x)

8.8 The volume V of liquid in a hollow horizontal cylinder of radius r and length L is related to the depth of the liquid h by

V 5 c r2cos21 ar 2 h r b 2 (r 2 h)22rh 2 h2 dL

Determine h given r 5 2 m, L 5 5 m, and V 5 8 m3. Note that if you are using a programming language or software tool that is not rich in trigonometric functions, the arc cosine can be computed with

cos21x 5 p

2 2 tan21 a x

21 2 x2 b

8.9 The volume V of liquid in a spherical tank of radius r is related to the depth h of the liquid by

V 5 ph2(3r 2 h)

3

Determine h given r 5 1 m and V 5 0.5 m3. 8.10 For the spherical tank in Prob. 8.9, it is possible to develop the following two fi xed-point formulas:

h 5 B

h3 1 (3Vyp) 3r

and

h 5 B

3 3 arh2 2 V p b

If r 5 1 m and V 5 0.5 m3, determine whether either of these is stable, and the range of initial guesses for which they are stable. 8.11 The operation of a constant density plug fl ow reactor for the production of a substance via an enzymatic reaction is described by the equation below, where V is the volume of the reactor, F is the fl ow rate of reactant C, Cin and Cout are the concentrations of reac- tant entering and leaving the reactor, respectively, and K and kmax are constants. For a 100-L reactor, with an inlet concentration of Cin 5 0.2 M, an inlet fl ow rate of 80 L /s, kmax 5 10

22 s21, and K 5 0.1 M, fi nd the concentration of C at the outlet of the reactor.

V

F 5 2#

Cout

Cin

K

kmaxC 1

1

kmax dC

8.12 The Ergun equation, shown below, is used to describe the fl ow of a fl uid through a packed bed. DP is the pressure drop, r is the density of the fl uid, Go is the mass velocity (mass fl ow rate di- vided by cross-sectional area), Dp is the diameter of the particles within the bed, m is the fl uid viscosity, L is the length of the bed, and e is the void fraction of the bed.

¢Pr G2o

Dp

L e3

1 2 e 5 150

1 2 e

(DpGoym) 1 1.75

cha9792x_ch08_204-225.indd Page 217 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 217 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

218 CASE STUDIES: ROOTS OF EQUATIONS

where the hyperbolic cosine can be computed by

cosh x 5 1

2 (ex 1 e2x)

Use a numerical method to calculate a value for the parameter TA given values for the parameters w 5 10 and y0 5 5, such that the cable has a height of y 5 15 at x 5 50. 8.18 Figure P8.18a shows a uniform beam subject to a linearly increasing distributed load. The equation for the resulting elastic curve is (see Fig. P8.18b)

y 5 w0

120EIL (2x5 1 2L2x3 2 L4x) (P8.18.1)

Use bisection to determine the point of maximum defl ection (that is, the value of x where dy/dx 5 0). Then substitute this value into Eq. (P8.18.1) to determine the value of the maximum defl ection. Use the following parameter values in your computation: L 5 450 cm, E 5 50,000 kN/cm2, I 5 30,000 cm4, and w0 5 1.75 kN/cm. 8.19 The displacement of a structure is defi ned by the following equation for a damped oscillation:

y 5 8e2kt cos vt

where k 5 0.5 and � 5 3. (a) Use the graphical method to make an initial estimate of the

time required for the displacement to decrease to 4. (b) Use the Newton-Raphson method to determine the root to

�s 5 0.01%. (c) Use the secant method to determine the root to �s 5 0.01%. 8.20 The Manning equation can be written for a rectangular open channel as

Q 5 1S(BH)5y3

n(B 1 2H)2y3

where x is the distance downstream in kilometers. (a) Determine the distance downstream where the oxygen level

fi rst falls to a reading of 5 mg/L. (Hint: It is within 2 km of the discharge.) Determine your answer to a 1% error. Note that levels of oxygen below 5 mg/L are generally harmful to game- fi sh such as trout and salmon.

(b) Determine the distance downstream at which the oxygen is at a minimum. What is the concentration at that location?

8.16 The concentration of pollutant bacteria c in a lake decreases according to

c 5 70e21.5t 1 25e20.075t

Determine the time required for the bacteria concentration to be reduced to 9 using (a) the graphical method and (b) using the Newton-Raphson method with an initial guess of t 5 10 and a stopping criterion of 0.5%. Check your result. 8.17 A catenary cable is one that is hung between two points not in the same vertical line. As depicted in Fig. P8.17a, it is subject to no loads other than its own weight. Thus, its weight (N/m) acts as a uniform load per unit length along the cable. A free-body diagram of a section AB is depicted in Fig. P8.17b, where TA and TB are the tension forces at the end. Based on horizontal and vertical force balances, the following differential equation model of the cable can be derived:

d2y

dx2 5

w

TAB 1 1 ady

dx b2

Calculus can be employed to solve this equation for the height y of the cable as a function of distance x,

y 5 TA w

cosh aw TA

xb 1 y0 2 TAw

FIGURE P8.17 (a) Forces acting on a section AB of a fl exible hanging cable. The load is uniform along the cable (but not uniform per the horizontal distance x). (b) A free- body diagram of section AB.

y

B

A

TA W = ws

w y0

x

(a) (b)

TB

cha9792x_ch08_204-225.indd Page 218 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 218 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 219

formula relating present worth P, annual payments A, number of years n, and interest rate i is

A 5 P i(1 1 i)n

(1 1 i)n 2 1

8.23 Many fi elds of engineering require accurate population esti- mates. For example, transportation engineers might fi nd it neces- sary to determine separately the population growth trends of a city and adjacent suburb. The population of the urban area is declining with time according to

Pu(t) 5 Pu, maxe 2kut 1 Pu, min

while the suburban population is growing, as in

Ps(t) 5 Ps, max

1 1 [Ps, maxyP0 2 1]e 2kst

where Pu, max, ku, Ps, max, P0, and ks 5 empirically derived parame- ters. Determine the time and corresponding values of Pu(t) and Ps(t) when the suburbs are 20% larger than the city. The parameter values are Pu, max 5 75,000, ku 5 0.045/yr, Pu, min 5 100,000 people, Ps, max 5 300,000 people, P0 5 10,000 people, ks 5 0.08/yr. To obtain your solutions, use (a) graphical, (b) false-position, and (c) modifi ed secant methods. 8.24 A simply supported beam is loaded as shown in Fig. P8.24. Using singularity functions, the shear along the beam can be expressed by the equation:

V(x) 5 20[kx 2 0l1 2 kx 2 5l1] 2 15 kx 2 8l0 2 57

By defi nition, the singularity function can be expressed as follows:

kx 2 aln 5 e (x 2 a)n when x . a 0 when x # a

f Use a numerical method to fi nd the point(s) where the shear equals zero.

where Q 5 fl ow [m3/s], S 5 slope [m/m], H 5 depth [m], and n  5 the Manning roughness coeffi cient. Develop a fi xed-point iteration scheme to solve this equation for H given Q 5 5, S 5 0.0002, B 5 20, and n 5 0.03. Prove that your scheme con- verges for all initial guesses greater than or equal to zero. 8.21 In ocean engineering, the equation for a refl ected standing wave in a harbor is given by � 5 16, t 5 12, � 5 48:

h 5 h0 c sin a2px l b cos a2pty

l b 1 e2x d

Solve for the lowest positive value of x if h 5 0.4h0. 8.22 You buy a $20,000 piece of equipment for nothing down and $4000 per year for 6 years. What interest rate are you paying? The

w0

L

(a)

(x = 0, y = 0) (x = L, y = 0)

x

(b)

FIGURE P8.18

FIGURE P8.24 20 kips/ft

150 kip-ft 15 kips

5’ 2’ 1’ 2’

cha9792x_ch08_204-225.indd Page 219 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 219 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

220 CASE STUDIES: ROOTS OF EQUATIONS

Electrical Engineering 8.29 Perform the same computation as in Sec. 8.3, but determine the value of L required for the circuit to dissipate to 1% of its origi- nal value in t 5 0.05 s, given R 5 280 V, and C 5 1024 F. Use (a) a graphical approach, (b) bisection, and (c) root location soft- ware such as the Excel Solver, the MATLAB function fzero, or the Mathcad function root. 8.30 An oscillating current in an electric circuit is described by i 5 9e2t sin(2 t), where t is in seconds. Determine the lowest value of t such that i 5 3.5. 8.31 The resistivity � of doped silicon is based on the charge q on an electron, the electron density n, and the electron mobility �. The electron density is given in terms of the doping density N  and the intrinsic carrier density ni. The electron mobility is described by the temperature T, the reference temperature T0, and the reference mobility �0. The equations required to com- pute the resistivity are

r 5 1

qnm

where

n 5 1

2 (N 1 2N2 1 4n2i )  and  m 5 m0 a TT0b

22.42

Determine N, given T0 5 300 K, T 5 1000 K, �0 5 1300 cm 2

(V s)21, q 5 1.6 3 10219 C, ni 5 6.21 3 10 9 cm23, and a desired

�  5 6 3 106 V s cm/C. Use (a) bisection and (b) the modifi ed secant method. 8.32 A total charge Q is uniformly distributed around a ring-shaped conductor with radius a. A charge q is located at a distance x from the center of the ring (Fig. P8.32). The force exerted on the charge by the ring is given by

F 5 1

4pe0

qQx

(x2 1 a2)3y2

where e0 5 8.85 3 10 212 C2/(N m2). Find the distance x where the

force is 1N if q and Q are 2 3 1025 C for a ring with a radius of 0.9 m.

8.25 Using the simply supported beam from Prob. 8.24, the mo- ment along the beam, M(x), is given by:

M(x) 5 210[kx 2 0l2 2 kx 2 5l2] 1 15 kx 2 8l1

1 150 kx 2 7l0 1 57x

Use a numerical method to fi nd the point(s) where the moment equals zero. 8.26 Using the simply supported beam from Prob. 8.24, the slope along the beam is given by:

duy

dx (x) 5

210

3 [kx 2 0l3 2 kx 2 5l3] 1

15

2 kx 2 8l2

1 150 kx 2 7l1 1 57

2 x2 2 238.25

Use a numerical method to fi nd the point(s) where the slope equals zero. 8.27 Using the simply supported beam from Prob. 8.24, the dis- placement along the beam is given by:

uy(x) 5 25

6 [kx 2 0l4 2 kx 2 5l4] 1

15

6 kx 2 8l3

1 75 kx 2 7l2 1 57

6 x3 2 238.25x

(a) Find the point(s) where the displacement equals zero. (b) How would you use a root location technique to determine the

location of the minimum displacement? 8.28 Although we did not mention it in Sec. 8.2, Eq. (8.10) is actu- ally an expression of electroneutrality; that is, that positive and negative charges must balance. This can be seen more clearly by expressing it as

[H1 ] 5 [HCO23 ] 1 2[CO 22 3 ] 1 [OH

2 ]

In other words, the positive charges must equal the negative charges. Thus, when you compute the pH of a natural water body such as a lake, you must also account for other ions that may be present. For the case where these ions originate from nonreactive salts, the net negative minus positive charges due to these ions are lumped together in a quantity called alkalinity, and the equation is reformulated as

Alk 1 [H1 ] 5 [HCO23 ] 1 2[CO 22 3 ] 1 [OH

2 ] (P8.28)

where Alk 5 alkalinity (eq/L). For example, the alkalinity of Lake Superior is approximately 0.4 3 1023 eq/L. Perform the same calculations as in Sec. 8.2 to compute the pH of Lake Superior in 2008. Assume that just like the raindrops, the lake is in equilib- rium with atmospheric CO2, but account for the alkalinity as in Eq. (P8.28).

FIGURE P8.32

x

a

Q

q

cha9792x_ch08_204-225.indd Page 220 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 220 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 221

8.36 Mechanical engineers, as well as most other engineers, use thermodynamics extensively in their work. The following polyno- mial can be used to relate the zero-pressure specifi c heat of dry air, cp kJ/(kg K), to temperature (K):

cp 5 0.99403 1 1.671 3 10 24T 1 9.7215 3 1028T 2

29.5838 3 10211T 3 1 1.9520 3 10214T 4

Determine the temperature that corresponds to a specifi c heat of 1.2 kJ/(kg K). 8.37 Aerospace engineers sometimes compute the trajectories of pro- jectiles like rockets. A related problem deals with the trajectory of a thrown ball. The trajectory of a ball is defi ned by the (x, y) coordinates, as displayed in Fig. P8.37. The trajectory can be modeled as

y 5 (tan u0)x 2 g

2y20 cos 2 u0

x 2 1 y0

Find the appropriate initial angle u0, if the initial velocity �0 5 20 m/s and the distance to the catcher x is 40 m. Note that the ball leaves the thrower’s hand at an elevation of y0 5 1.8 m and the catcher receives it at 1 m. Express the fi nal result in degrees. Use a value of 9.81 m/s2 for g and employ the graphical method to develop your initial guesses.

8.33 Figure P8.33 shows a circuit with a resistor, an inductor, and a capacitor in parallel. Kirchhoff’s rules can be used to express the impedance of the system as

1

Z 5 B

1

R2 1 avC 2 1

vL b2

where Z 5 impedance (V) and v 5 the angular frequency. Find the � that results in an impedance of 75 V using both bisection and false position with initial guesses of 1 and 1000 for the following parameters: R 5 225 V, C 5 0.6 3 1026 F, and L 5 0.5 H. Deter- mine how many iterations of each technique are necessary to deter- mine the answer to �s 5 0.1%. Use the graphical approach to explain any diffi culties that arise.

FIGURE P8.33

R L C�

FIGURE P8.35

h

(a) (b)

d h + d

Mechanical and Aerospace Engineering 8.34 Beyond the Colebrook equation, other relationships, such as the Fanning friction factor f, are available to estimate friction in pipes. The Fanning friction factor is dependent on a number of pa- rameters related to the size of the pipe and the fl uid, which can all be represented by another dimensionless quantity, the Reynolds number Re. A formula that predicts f given Re is the von Karman equation,

1

1f 5 4 log10(Re1f ) 2 0.4

Typical values for the Reynolds number for turbulent fl ow are 10,000 to 500,000 and for the Fanning friction factor are 0.001 to 0.01. De- velop a function that uses bisection to solve for f given a user-supplied value of Re between 2500 and 1,000,000. Design the function so that it ensures that the absolute error in the result is Ea,d , 0.000005. 8.35 Real mechanical systems may involve the defl ection of nonlin- ear springs. In Fig. P8.35, a mass m is released a distance h above a nonlinear spring. The resistance force F of the spring is given by

F 5 2(k1d 1 k2d 3y2)

Conservation of energy can be used to show that

0 5 2k2d

5y2

5 1

1

2 k1d

2 2 mgd 2 mgh

Solve for d, given the following parameter values: k1 5 40,000 g/s 2,

k2 5 40 g/(s 2 m0.5), m 5 95 g, g 5 9.81 m/s2, and h 5 0.43 m.

FIGURE P8.37

�0

v0

y

x

cha9792x_ch08_204-225.indd Page 221 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 221 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

222 CASE STUDIES: ROOTS OF EQUATIONS

As a mechanical engineer, you would like to know if there are cases where 5 �y2 2 1. Use the other parameters from the section to set up the equation as a roots problem and solve for �. 8.41 Two fl uids at different temperatures enter a mixer and come out at the same temperature. The heat capacity of fl uid A is given by:

cp 5 3.381 1 1.804 3 10 22T 2 4.300 3 1026T 2

and the heat capacity of fl uid B is given by:

cp 5 8.592 1 1.290 3 10 21T 2 4.078 3 1025T 2

where cp is in units of cal/mol K, and T is in units of K. Note that

¢H 5 # T2

T1

cpdT

A enters the mixer at 4008C. B enters the mixer at 6008C. There is twice as much A as there is B entering into the mixer. At what tem- perature do the two fl uids exit the mixer? 8.42 A compressor is operating at compression ratio Rc of 3.0 (the pressure of gas at the outlet is three times greater than the pressure of the gas at the inlet). The power requirements of the compressor Hp can be determined from the equation below. Assuming that the power requirements of the compressor are exactly equal to zRT1yMW, fi nd the polytropic effi ciency n of the compressor. The parameter z is compressibility of the gas under operating condi- tions of the compressor, R is the gas constant, T1 is the temperature of the gas at the compressor inlet, and MW is the molecular weight of the gas.

HP 5 z RT1 MW

n

n 2 1 (R(n21)ync 2 1)

8.43 In the thermos shown in Fig. P8.43, the innermost compart- ment is separated from the middle container by a vacuum. There is a fi nal shell around the thermos. This fi nal shell is separated from the middle layer by a thin layer of air. The outside of the fi nal shell comes in contact with room air. Heat transfer from the inner compartment to the next layer q1 is by radiation only (since the space is evacuated). Heat transfer between the middle layer and outside shell q2 is by convection in a small space. Heat trans- fer from the outside shell to the air q3 is by natural convection. The heat fl ux from each region of the thermos must be equal— that is, q1 5 q2 5 q3. Find the temperatures T1 and T2 at steady state. T0 is 5008C and T3 5 258C.

q1 5 10 29[ (T0 1 273)

4 2 (T1 1 273) 4]

q2 5 4(T1 2 T2)

q3 5 1.3(T2 2 T3) 4y3

8.38 The general form for a three-dimensional stress fi eld is given by

£ sxx sxy sxz

sxy syy syz

sxz syz szz

§

where the diagonal terms represent tensile or compressive stresses and the off-diagonal terms represent shear stresses. A stress fi eld (in MPa) is given by

£ 10 14 25

14 7 15

25 15 16

§

To solve for the principal stresses, it is necessary to construct the following matrix (again in MPa):

£ 10 2 s 14 25

14 7 2 s 15

25 15 16 2 s

§

�1, �2, and �3 can be solved from the equation

s3 2 Is2 1 IIs 2 III 5 0

where

I 5 sxx 1 syy 1 szz

II 5 sxxsyy 1 sxxszz 1 syyszz 2 s 2 xy 2 s

2 xz 2 s

2 yz

III 5 sxxsyyszz 2 sxxs 2 yz 2 syys

2 xz 2 szzs

2 xy 1 2sxysxzsyz

I, II, and III are known as the stress invariants. Find �1, �2, and �3 using a root-fi nding technique. 8.39 The upward velocity of a rocket can be computed by the fol- lowing formula:

y 5 u ln m0

m0 2 qt 2 gt

where � 5 upward velocity, u 5 the velocity at which fuel is ex- pelled relative to the rocket, m0 5 the initial mass of the rocket at time t 5 0, q 5 the fuel consumption rate, and g 5 the downward ac- celeration of gravity (assumed constant 5 9.81 m/s2). If u 5 2200 m/s, m0 5 160,000 kg, and q 5 2680 kg/s, compute the time at which � 5 1000 m/s. (Hint: t is somewhere between 10 and 50 s.) Determine your result so that it is within 1% of the true value. Check your answer. 8.40 The phase angle between the forced vibration caused by the rough road and the motion of the car is given by

tan f 5 2(cycc) (vyp)

1 2 (vyp)2

cha9792x_ch08_204-225.indd Page 222 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 222 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 223

8.45 A fl uid is pumped into the network of pipes shown in Fig. P8.45. At steady state, the following fl ow balances must hold,

Q1 5 Q2 1 Q3

Q3 5 Q4 1 Q5

Q5 5 Q6 1 Q7

where Qi 5 fl ow in pipe i(m 3/s). In addition, the pressure drops

around the three right-hand loops must equal zero. The pressure drop in each circular pipe length can be computed with

¢P 5 16

p2 f Lr

2 D5 Q2

where DP 5 the pressure drop (Pa), f 5 the friction factor (dimen- sionless), L 5 the pipe length (m), � 5 the fl uid density (kg/m3), and D 5 pipe diameter (m). Write a program (or develop an algo- rithm in a mathematics software package) that will allow you to compute the fl ow in every pipe length given that Q1 5 1 m

3/s and � 5 1.23 kg/m3. All the pipes have D 5 500 mm and f 5 0.005. The pipe lengths are: L3 5 L5 5 L8 5 L9 5 2 m; L2 5 L4 5 L6 5 4 m; and L7 5 8 m. 8.46 Repeat Prob. 8.45, but incorporate the fact that the friction factor can be computed with the von Karman equation,

1

1f 5 4 log10(Re1f ) 2 0.4

where Re 5 the Reynolds number

Re 5 rVD

m

where V 5 the velocity of the fl uid in the pipe (m/s) and � 5 dynamic viscosity (N ? s/m2). Note that for a circular pipe

8.44 Figure P8.44 shows three reservoirs connected by circular pipes. The pipes, which are made of asphalt-dipped cast iron (ε 5 0.0012 m), have the following characteristics:

Pipe 1 2 3 Length, m 1800 500 1400 Diameter, m 0.4 0.25 0.2 Flow, m3/s ? 0.1 ?

If the water surface elevations in Reservoirs A and C are 200 and 172.5 m, respectively, determine the elevation in Reservoir B and the fl ows in pipes 1 and 3. Note that the kinematic viscosity of water is 1 3 1026 m2/s and use the Colebrook equation to deter- mine the friction factor (recall Prob. 8.13).

FIGURE P8.43

T0

T2

T3

T1 FIGURE P8.45

Q1

Q10 Q9 Q8

Q3 Q5

Q7Q6Q4Q2

FIGURE P8.44

Q1

h2

h3

h1

Q3

Q2

1 2

3

A

B

C

cha9792x_ch08_204-225.indd Page 223 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 223 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

224 CASE STUDIES: ROOTS OF EQUATIONS

V 5 4Q/ D2. Also, assume that the fl uid has a viscosity of 1.79 3 1025 N ? s/m2. 8.47 The space shuttle, at lift-off from the launch pad, has four forces acting on it, which are shown on the free-body diagram (Fig. P8.47). The combined weight of the two solid rocket boost- ers and external fuel tank is WB 5 1.663 3 10

6 lb. The weight of

FIGURE P8.47

External tank

Solid rocket booster

Orbiter

38’

4’

28’

WB WS

TS

TB

G

the orbiter with a full payload is WS 5 0.23 3 10 6 lb. The com-

bined thrust of the two solid rocket boosters is TB 5 5.30 3 10 6 lb.

The combined thrust of the three liquid fuel orbiter engines is TS 5 1.125 3 106 lb.

At liftoff, the orbiter engine thrust is directed at angle � to make the resultant moment acting on the entire craft assembly (external tank, solid rocket boosters, and orbiter) equal to zero. With the resultant moment equal to zero, the craft will not rotate about its mass center G at liftoff. With these forces, the craft will have a resultant force with components in both the vertical and horizontal direction. The vertical resultant force component is what allows the craft to lift off from the launch pad and fl y verti- cally. The horizontal resultant force component causes the craft to fl y horizontally. The resultant moment acting on the craft will be zero when � is adjusted to the proper value. If this angle is not adjusted properly, and there is some resultant moment acting on the craft, the craft will tend to rotate about it mass center. (a) Resolve the orbiter thrust TS into horizontal and vertical com-

ponents, and then sum moments about point G, the craft mass center. Set the resulting moment equation equal to zero. This equation can now be solved for the value of � required for liftoff.

(b) Derive an equation for the resultant moment acting on the craft in terms of the angle �. Plot the resultant moment as a function of the angle � over a range of 25 radians to 15 radians.

(c) Write a computer program to solve for the angle � using Newton’s method to fi nd the root of the resultant moment equa- tion. Make an initial fi rst guess at the root of interest using the plot. Terminate your iterations when the value of � has better than fi ve signifi cant fi gures.

(d) Repeat the program for the minimum payload weight of the orbiter of WS 5 195,000 lb.

8.48 Determining the velocity of particles settling through fl uids is of great importance of many areas of engineering and science. Such calculations depend on the fl ow regime as represented by the dimensionless Reynolds number,

Re 5 rdy

m (P8.48.1)

where � 5 the fl uid’s density (kg/m3), d 5 the particle diameter (m), y 5 the particle’s settling velocity (m/s), and � 5 the fl uid’s dynamic viscosity (N s/m2). Under laminar conditions (Re , 0.1), the settling velocity of a spherical particle can be computed with the following formula based on Stokes law,

y 5 g

18 ars 2 r m b d2 (P8.48.2)

where g 5 the gravitational constant (5 9.81 m/s2), and �s 5 the particle’s density (kg/m3). For turbulent conditions (i.e., higher

cha9792x_ch08_204-225.indd Page 224 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 224 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 225

(b) Use the modifi ed secant method with d 5 1023 and εS 5 0.05% to determine y for a spherical iron particle settling in water, where d 5 200 �m, � 5 1 g/cm3, �s 5 7.874 g/cm

3, and � 5 0.014 g/(cm?s). Employ Eq. (P8.48.2) to generate your initial guess.

(c) Based on the result of (b), compute the Reynolds number and the drag coeffi cient, and use the latter to confi rm that the fl ow regime is not laminar.

(d) Develop a fi xed-point iteration solution for the conditions outlined in (b).

(e) Use a graphical approach to illustrate that the formulation developed in (d) will converge for any positive guess.

Reynolds numbers), an alternative approach can be used based on the following formula:

y 5 B

4g(rs 2 r)d

3CDr (P8.48.3)

where CD 5 the drag coeffi cient, which depends on the Reynolds number as in

CD 5 24

Re 1

3

1Re 1 0.34 (P8.48.4)

(a) Combine Eqs. (P8.48.2), (P8.48.3), and (P8.48.4) to express the determination of y as a roots of equations problem. That is, express the combined formula in the format f(y) 5 0.

cha9792x_ch08_204-225.indd Page 225 15/10/13 5:28 PM f-479 cha9792x_ch08_204-225.indd Page 225 15/10/13 5:28 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

226

PT2.4 TRADE-OFFS

Table PT2.3 provides a summary of the trade-offs involved in solving for roots of alge- braic and transcendental equations. Although graphical methods are time-consuming, they provide insight into the behavior of the function and are useful in identifying initial guesses and potential problems such as multiple roots. Therefore, if time permits, a quick sketch (or better yet, a computerized graph) can yield valuable information regarding the behavior of the function. The numerical methods themselves are divided into two general categories: bracket- ing and open methods. The former requires two initial guesses that are on either side of a root. This “bracketing” is maintained as the solution proceeds, and thus, these tech- niques are always convergent. However, a price is paid for this property in that the rate of convergence is relatively slow.

TABLE PT2.3 Comparison of the characteristics of alternative methods for fi nding roots of algebraic and transcendental equations. The comparisons are based on general experience and do not account for the behavior of specifi c functions.

Method Type Guesses Convergence Stability Programming Comments

Direct Analytical — — — Graphical Visual — — — — Imprecise Bisection Bracketing 2 Slow Always Easy False-position Bracketing 2 Slow/medium Always Easy Modifi ed FP Bracketing 2 Medium Always Easy Fixed-point Open 1 Slow Possibly divergent Easy iteration Newton-Raphson Open 1 Fast Possibly divergent Easy Requires evaluation of f’(x) Modifi ed Newton- Open 1 Fast (multiple), Possibly divergent Easy Requires Raphson medium (single) evaluation of f’(x) and f”(x) Secant Open 2 Medium/fast Possibly divergent Easy Initial guesses do not have to bracket the root Modifi ed secant Open 1 Medium/fast Possibly divergent Easy Brent Hybrid 1 or 2 Medium Always (for Moderate Robust 2 guesses) Müller Polynomials 2 Medium/fast Possibly divergent Moderate Bairstow Polynomials 2 Fast Possibly divergent Moderate

EPILOGUE: PART TWO

cha9792x_ep02_226-229.indd Page 226 06/10/13 5:52 PM F-468 cha9792x_ep02_226-229.indd Page 226 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT2.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 227

Open techniques differ from bracketing methods in that they use information at a single point (or two values that need not bracket the root to extrapolate to a new root estimate). This property is a double-edged sword. Although it leads to quicker conver- gence, it also allows the possibility that the solution may diverge. In general, the con- vergence of open techniques is partially dependent on the quality of the initial guess and the nature of the function. The closer the guess is to the true root, the more likely the methods will converge. Of the open techniques, the standard Newton-Raphson method is often used because of its property of quadratic convergence. However, its major shortcoming is that it re- quires the derivative of the function be obtained analytically. For some functions this is impractical. In these cases, the secant method, which employs a fi nite-difference repre- sentation of the derivative, provides a viable alternative. Because of the fi nite-difference approximation, the rate of convergence of the secant method is initially slower than for the Newton-Raphson method. However, as the root estimate is refi ned, the difference approximation becomes a better representation of the true derivative, and convergence accelerates rapidly. The modifi ed Newton-Raphson technique can be used to attain rapid convergence for multiple roots. However, this technique requires an analytical expression for both the fi rst and second derivatives. Of particular interest are hybrid methods that combine the reliability of bracketing with the speed of open methods. Brent’s method does this by combining bisection with several open methods. All the methods are easy-to-moderate to program on computers and require minimal time to determine a single root. On this basis, you might conclude that simple methods such as bisection would be good enough for practical purposes. This would be true if you were exclusively interested in determining the root of an equation once. However, there are many cases in engineering where numerous root locations are required and where speed becomes important. For these cases, slow meth- ods are very time-consuming and, hence, costly. On the other hand, the fast open meth- ods may diverge, and the accompanying delays can also be costly. Some computer algorithms attempt to capitalize on the strong points of both classes of techniques by initially employing a bracketing method to approach the root, then switching to an open method to rapidly refi ne the estimate. Whether a single approach or a combination is used, the trade-offs between convergence and speed are at the heart of the choice of a root-location technique.

PT2.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT2.4 summarizes important information that was presented in Part Two. This table can be consulted to quickly access important relationships and formulas.

PT2.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

The methods in this text have focused on determining a single real root of an algebraic or transcendental equation based on foreknowledge of its approximate location. In ad- dition, we have also described methods expressly designed to determine both the real

cha9792x_ep02_226-229.indd Page 227 06/10/13 5:52 PM F-468 cha9792x_ep02_226-229.indd Page 227 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

228 EPILOGUE: PART TWO

and complex roots of polynomials. Additional references on the subject are Ralston and Rabinowitz (1978) and Carnahan, Luther, and Wilkes (1969). In addition to Müller’s and Bairstow’s methods, several techniques are available to determine all the roots of polynomials. In particular, the quotient difference (QD) algo- rithm (Henrici, 1964, and Gerald and Wheatley, 2004) determines all roots without initial guesses. Ralston and Rabinowitz (1978) and Carnahan, Luther, and Wilkes (1969)

TABLE PT2.4 Summary of important information presented in Part Two.

Graphical Errors and Method Formulation Interpretation Stopping Criteria

Bracketing methods:

Bisection xr 5 xl 1 xu

2 Stopping criterion:

If f (xl)f (xr) , 0, xu 5 xr ` x new r 2 x oldr

x newr ` 100% # es

f (xl)f (xr) . 0, xl 5 xr

False position xr 5 xu 2 f (xu) (xl 2 xu) f (xl) 2 f (xu)

Stopping criterion:

If f (xl)f (xr) , 0, xu 5 xr ` x new r 2 xoldr

x newr ` 100% # es

f (xl)f (xr) . 0, xl 5 xr

Newton-Raphson Stopping criterion:

xi11 5 xi 2 f (xi) f ¿ (xi)

` xi11 2 xi xi11

` 100% # es Error: Ei11 5 0(E2i )

Secant Stopping criterion:

xi11 5 xi 2 f (xi) (xi21 2 xi) f (xi21) 2 f (xi)

` xi11 2 xi xi11

` 100% # es

f (x)

xxuxl L

L/2

Root

L/4

f (x)

xxuxl

xr Ch

or d

f (x)

xxixi + 1

Tangent

f (x)

xxi xi – 1 xi + 1

cha9792x_ep02_226-229.indd Page 228 06/10/13 5:52 PM F-468 cha9792x_ep02_226-229.indd Page 228 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT2.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 229

contain discussions of this method as well as of other techniques for locating roots of polynomials. As discussed in the text, the Jenkins-Traub and Laguerre’s methods are widely employed. In summary, the foregoing is intended to provide you with avenues for deeper exploration of the subject. Additionally, all the above references provide descrip- tions of the basic techniques covered in Part Two. We urge you to consult these alternative sources to broaden your understanding of numerical methods for root location.1

1Books are referenced only by author here, a complete bibliography will be found at the back of this text.

cha9792x_ep02_226-229.indd Page 229 06/10/13 5:52 PM F-468 cha9792x_ep02_226-229.indd Page 229 06/10/13 5:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART THREE

cha9792x_p03_230-244.indd Page 230 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 230 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

231

PT3.1 MOTIVATION

In Part Two, we determined the value x that satisfi ed a single equation, f(x) 5 0. Now, we deal with the case of determining the values x1, x2, . . . , xn that simultaneously sat- isfy a set of equations

f1(x1, x2, p , xn) 5 0 f2(x1, x2, p , xn) 5 0 . . . . . .

fn(x1, x2, p , xn) 5 0

Such systems can be either linear or nonlinear. In Part Three, we deal with linear alge- braic equations that are of the general form

a11x1 1 a12x2 1 p 1 a1nxn 5 b1 a21x1 1 a22x2 1 p 1 a2nxn 5 b2 . . . . (PT3.1) . .

an1x1 1 an2x2 1 p 1 annxn 5 bn where the a’s are constant coeffi cients, the b’s are constants, and n is the number of equa- tions. All other equations are nonlinear. Nonlinear systems were discussed in Chap. 6 and will be covered briefl y again in Chap. 9.

PT3.1.1 Noncomputer Methods for Solving Systems of Equations

For small numbers of equations (n # 3), linear (and sometimes nonlinear) equations can be solved readily by simple techniques. Some of these methods will be reviewed at the beginning of Chap. 9. However, for four or more equations, solutions become arduous and computers must be utilized. Historically, the inability to solve all but the smallest sets of equations by hand has limited the scope of problems addressed in many engineering applications. Before computers, techniques to solve linear algebraic equations were time-consum- ing and awkward. These approaches placed a constraint on creativity because the methods were often diffi cult to implement and understand. Consequently, the techniques were sometimes overemphasized at the expense of other aspects of the problem-solving process such as formulation and interpretation (recall Fig. PT1.1 and accompanying discussion).

LINEAR ALGEBRAIC EQUATIONS

cha9792x_p03_230-244.indd Page 231 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 231 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

232 LINEAR ALGEBRAIC EQUATIONS

The advent of easily accessible computers makes it possible and practical for you to solve large sets of simultaneous linear algebraic equations. Thus, you can approach more complex and realistic examples and problems. Furthermore, you will have more time to test your creative skills because you will be able to place more emphasis on problem formulation and solution interpretation.

PT3.1.2 Linear Algebraic Equations and Engineering Practice

Many of the fundamental equations of engineering are based on conservation laws (recall Table 1.1). Some familiar quantities that conform to such laws are mass, energy, and momentum. In mathematical terms, these principles lead to balance or continuity equa- tions that relate system behavior as represented by the levels or response of the quantity being modeled to the properties or characteristics of the system and the external stimuli or forcing functions acting on the system. As an example, the principle of mass conservation can be used to formulate a model for a series of chemical reactors (Fig. PT3.1a). For this case, the quantity being modeled is the mass of the chemical in each reactor. The system properties are the reaction char- acteristics of the chemical and the reactors’ sizes and fl ow rates. The forcing functions are the feed rates of the chemical into the system. In Part Two, you saw how single-component systems result in a single equation that can be solved using root-location techniques. Multicomponent systems result in a coupled set of mathematical equations that must be solved simultaneously. The equations are

FIGURE PT3.1 Two types of systems that can be modeled using linear algebraic equations: (a) lumped variable system that involves coupled fi nite components and (b) distributed variable system that involves a continuum.

x1 x1 xi�1xi�1 xn

(b)

Feed

Feed x1 x5

(a)

……

x2

x3

x4

cha9792x_p03_230-244.indd Page 232 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 232 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.2 MATHEMATICAL BACKGROUND 233

coupled because the individual parts of the system are infl uenced by other parts. For example, in Fig. PT3.1a, reactor 4 receives chemical inputs from reactors 2 and 3. Con- sequently, its response is dependent on the quantity of chemical in these other reactors. When these dependencies are expressed mathematically, the resulting equations are often of the linear algebraic form of Eq. (PT3.1). The x’s are usually measures of the magnitudes of the responses of the individual components. Using Fig. PT3.1a as an example, x1 might quantify the amount of mass in the fi rst reactor, x2 might quantify the amount in the second, and so forth. The a’s typically represent the properties and char- acteristics that bear on the interactions between components. For instance, the a’s for Fig. PT3.1a might be refl ective of the fl ow rates of mass between the reactors. Finally, the b’s usually represent the forcing functions acting on the system, such as the feed rate in Fig. PT3.1a. The applications in Chap. 12 provide other examples of such equations derived from engineering practice. Multicomponent problems of the above types arise from both lumped (macro-) or distributed (micro-) variable mathematical models (Fig. PT3.1). Lumped variable prob- lems involve coupled fi nite components. Examples include trusses (Sec. 12.2), reactors (Fig. PT3.1a and Sec. 12.1), and electric circuits (Sec. 12.3). These types of problems use models that provide little or no spatial detail. Conversely, distributed variable problems attempt to describe spatial detail of sys- tems on a continuous or semicontinuous basis. The distribution of chemicals along the length of an elongated, rectangular reactor (Fig. PT3.1b) is an example of a continuous variable model. Differential equations derived from the conservation laws specify the distribution of the dependent variable for such systems. These differential equations can be solved numerically by converting them to an equivalent system of simultaneous alge- braic equations. The solution of such sets of equations represents a major engineering application area for the methods in the following chapters. These equations are coupled because the variables at one location are dependent on the variables in adjoining regions. For example, the concentration at the middle of the reactor is a function of the concen- tration in adjoining regions. Similar examples could be developed for the spatial distribu- tion of temperature or momentum. We will address such problems when we discuss differential equations later in the book. Aside from physical systems, simultaneous linear algebraic equations also arise in a variety of mathematical problem contexts. These result when mathematical functions are required to satisfy several conditions simultaneously. Each condition results in an equation that contains known coeffi cients and unknown variables. The techniques dis- cussed in this part can be used to solve for the unknowns when the equations are linear and algebraic. Some widely used numerical techniques that employ simultaneous equa- tions are regression analysis (Chap. 17) and spline interpolation (Chap. 18).

PT3.2 MATHEMATICAL BACKGROUND

All parts of this book require some mathematical background. For Part Three, matrix notation and algebra are useful because they provide a concise way to represent and manipulate linear algebraic equations. If you are already familiar with matrices, feel free to skip to Sec. PT3.3. For those who are unfamiliar or require a review, the following material provides a brief introduction to the subject.

cha9792x_p03_230-244.indd Page 233 18/12/13 9:38 AM F-468 cha9792x_p03_230-244.indd Page 233 18/12/13 9:38 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

234 LINEAR ALGEBRAIC EQUATIONS

PT3.2.1 Matrix Notation

A matrix consists of a rectangular array of elements represented by a single symbol. As depicted in Fig. PT3.2, [A] is the shorthand notation for the matrix and aij designates an individual element of the matrix. A horizontal set of elements is called a row and a vertical set is called a column. The fi rst subscript i always designates the number of the row in which the element lies. The second subscript j designates the column. For example, element a23 is in row 2 and column 3. The matrix in Fig. PT3.2 has n rows and m columns and is said to have a dimension of n by m (or n 3 m). It is referred to as an n by m matrix. Matrices with row dimension n 5 1, such as

[B] 5 [b1 b2 p bm]

are called row vectors. Note that for simplicity, the fi rst subscript of each element is dropped. Also, it should be mentioned that there are times when it is desirable to employ a special shorthand notation to distinguish a row matrix from other types of matrices. One way to accomplish this is to employ special open-topped brackets, as in :B ; . Matrices with column dimension m 5 1, such as

[C] 5 Fc1c2. .

.

cn

V are referred to as column vectors. For simplicity, the second subscript is dropped. As with the row vector, there are occasions when it is desirable to employ a special short- hand notation to distinguish a column matrix from other types of matrices. One way to accomplish this is to employ special brackets, as in {C}.

FIGURE PT3.2 A matrix.

Column 3

[A] 5 Fa11 a12 a13 p a1ma21 a22 a23 p a2m. . . . . .

. . .

an1 an2 an3 p anm

V Row 2

cha9792x_p03_230-244.indd Page 234 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 234 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.2 MATHEMATICAL BACKGROUND 235

Matrices where n 5 m are called square matrices. For example, a 4 by 4 matrix is

[A] 5 ≥ a11 a12 a13 a14a21 a22 a23 a24 a31 a32 a33 a34 a41 a42 a43 a44

¥ The diagonal consisting of the elements a11, a22, a33, and a44 is termed the principal or main diagonal of the matrix. Square matrices are particularly important when solving sets of simultaneous linear equations. For such systems, the number of equations (corresponding to rows) and the number of unknowns (corresponding to columns) must be equal for a unique solution to be possible. Consequently, square matrices of coeffi cients are encountered when dealing with such systems. Some special types of square matrices are described in Box PT3.1.

There are a number of special forms of square matrices that are important and should be noted: A symmetric matrix is one where aij 5 aji for all i’s and j’s. For example,

[A] 5 £ 5 1 21 3 7 2 7 8

§ is a 3 by 3 symmetric matrix. A diagonal matrix is a square matrix where all elements off the main diagonal are equal to zero, as in

[A] 5 ≥ a11 a22 a33

a44

¥ Note that where large blocks of elements are zero, they are left blank. An identity matrix is a diagonal matrix where all elements on the main diagonal are equal to 1, as in

[I] 5 ≥ 1 1 1 1

¥

The symbol [I] is used to denote the identity matrix. The identity matrix has properties similar to unity. An upper triangular matrix is one where all the elements below the main diagonal are zero, as in

[A] 5 ≥ a11 a12 a13 a14a22 a23 a24 a33 a34

a44

¥ A lower triangular matrix is one where all elements above the main diagonal are zero, as in

[A] 5 ≥ a11a21 a22 a31 a32 a33 a41 a42 a43 a44

¥ A banded matrix has all elements equal to zero, with the excep- tion of a band centered on the main diagonal:

[A] 5 ≥ a12 a12a21 a22 a23 a32 a33 a34

a43 a44

¥ The above matrix has a bandwidth of 3 and is given a special name—the tridiagonal matrix.

Box PT3.1 Special Types of Square Matrices

cha9792x_p03_230-244.indd Page 235 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 235 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

236 LINEAR ALGEBRAIC EQUATIONS

PT3.2.2 Matrix Operating Rules

Now that we have specifi ed what we mean by a matrix, we can defi ne some operating rules that govern its use. Two n by m matrices are equal if, and only if, every element in the fi rst is equal to every element in the second, that is, [A] 5 [B] if aij 5 bij for all i and j. Addition of two matrices, say, [A] and [B], is accomplished by adding corresponding terms in each matrix. The elements of the resulting matrix [C] are computed,

cij 5 aij 1 bij

for i 5 1, 2, . . . , n and j 5 1, 2, . . . , m. Similarly, the subtraction of two matrices, say, [E] minus [F], is obtained by subtracting corresponding terms, as in

dij 5 eij 2 fij

for i 5 1, 2, . . . , n and j 5 1, 2, . . . , m. It follows directly from the above defi nitions that addition and subtraction can be performed only between matrices having the same dimensions. Both addition and subtraction are commutative:

[A] 1 [B] 5 [B] 1 [A]

Addition and subtraction are also associative, that is,

( [A] 1 [B]) 1 [C] 5 [A] 1 ([B] 1 [C])

The multiplication of a matrix [A] by a scalar g is obtained by multiplying every element of [A] by g, as in

[D] 5 g[A] 5 F ga11 ga12 p ga1mga21 ga22 p ga2m. . . . . .

. . .

gan1 gan2 p ganm

V The product of two matrices is represented as [C] 5 [A][B], where the elements of [C] are defi ned as (see Box PT3.2 for a simple way to conceptualize matrix multiplication)

cij 5 a n

k51 aikbkj (PT3.2)

where n 5 the column dimension of [A] and the row dimension of [B]. That is, the cij element is obtained by adding the product of individual elements from the ith row of the fi rst matrix, in this case [A], by the jth column of the second matrix [B]. According to this defi nition, multiplication of two matrices can be performed only if the fi rst matrix has as many columns as the number of rows in the second matrix. Thus, if [A] is an n by m matrix, [B] could be an m by l matrix. For this case, the result- ing [C] matrix would have the dimension of n by l. However, if [B] were an l by m matrix, the multiplication could not be performed. Figure PT3.3 provides an easy way to check whether two matrices can be multiplied.

cha9792x_p03_230-244.indd Page 236 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 236 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.2 MATHEMATICAL BACKGROUND 237

FIGURE PT3.3

Box PT3.2 A Simple Method for Multiplying Two Matrices

Although Eq. (PT3.2) is well suited for implementation on a computer, it is not the simplest means for visualizing the mechanics of multiplying two matrices. What follows gives more tangible expression to the operation. Suppose that we want to multiply [X] by [Y] to yield [Z],

[Z] 5 [X] [Y] 5 £ 3 18 6 0 4

§ c 5 9 7 2

d A simple way to visualize the computation of [Z] is to raise [Y], as in

A

c 5 9 7 2

d d [Y] [X] S £ 3 18 6

0 4

§ £ ?  § d [Z] Now the answer [Z] can be computed in the space vacated by [Y]. This format has utility because it aligns the appropriate rows and columns that are to be multiplied. For example, according to Eq. (PT3.2), the element z11 is obtained by multiplying the fi rst row of [X] by the fi rst column of [Y]. This amounts to adding the product of x11 and y11 to the product of x12 and y21, as in

c 5 9 7 2

d T

£ 3 18 6 0 4

§ S £ 3 3 5 1 1 3 7 5 22   § Thus, z11 is equal to 22. Element z21 can be computed in a similar fashion, as in c 5 9

7 2 d

T

£ 3 18 6 0 4

§ S £ 228 3 5 1 6 3 7 5 82   § The computation can be continued in this way, following the alignment of the rows and columns, to yield the result

[Z] 5 £ 22 2982 84 28 8

§ Note how this simple method makes it clear why it is impossible to multiply two matrices if the number of columns of the fi rst ma- trix does not equal the number of rows in the second matrix. Also, note how it demonstrates that the order of multiplication matters (that is, matrix multiplication is not commutative).

[A]n � m [B]m � l � [C]n � l Interior dimensions

are equal; multiplication is possible

Exterior dimensions define the dimensions of the result

cha9792x_p03_230-244.indd Page 237 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 237 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

238 LINEAR ALGEBRAIC EQUATIONS

If the dimensions of the matrices are suitable, matrix multiplication is associative,

([A][B])[C] 5 [A]([B][C])

and distributive,

[A]([B] 1 [C]) 5 [A][B] 1 [A][C]

or

([A] 1 [B])[C] 5 [A][C] 1 [B][C]

However, multiplication is not generally commutative:

[A][B] ? [B][A]

That is, the order of multiplication is important. Figure PT3.4 shows pseudocode to multiply an n by m matrix [A], by an m by l matrix [B], and store the result in an n by l matrix [C]. Notice that, instead of the inner product being directly accumulated in [C], it is collected in a temporary vari- able, sum. This is done for two reasons. First, it is a bit more effi cient, because the computer need determine the location of ci, j only n 3 l times rather than n 3 l 3 m times. Second, the precision of the multiplication can be greatly improved by declar- ing sum as a double precision variable (recall the discussion of inner products in Sec. 3.4.2). Although multiplication is possible, matrix division is not a defi ned operation. How- ever, if a matrix [A] is square and nonsingular, there is another matrix [A]21, called the inverse of [A], for which

[A][A]21 5 [A]21[A] 5 [I] (PT3.3)

Thus, the multiplication of a matrix by the inverse is analogous to division, in the sense that a number divided by itself is equal to 1. That is, multiplication of a matrix by its inverse leads to the identity matrix (recall Box PT3.1). The inverse of a two-dimensional square matrix can be represented simply by

[A]21 5 1

a11a22 2 a12a21 c a22 2a12

2a21 a11 d (PT3.4)

SUBROUTINE Mmult (a, b, c, m, n, l) DOFOR i 5 1, n DOFOR j 5 1, l sum 5 0. DOFOR k 5 1, m sum 5 sum 1 a(i,k) ? b(k,j) END DO c(i,j) 5 sum END DO END DO

FIGURE PT3.4

cha9792x_p03_230-244.indd Page 238 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 238 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.2 MATHEMATICAL BACKGROUND 239

Similar formulas for higher-dimensional matrices are much more involved. Sections in Chaps. 10 and 11 will be devoted to techniques for using numerical methods and the computer to calculate the inverse for such systems. Two other matrix manipulations that will have utility in our discussion are the trans- pose and the trace of a matrix. The transpose of a matrix involves transforming its rows into columns and its columns into rows. For example, for the 4 3 4 matrix,

[A] 5 ≥ a11 a12 a13 a14a21 a22 a23 a24a31 a32 a33 a34 a41 a42 a43 a44

¥ the transpose, designated [A]T, is defi ned as

[A]T 5 ≥ a11 a21 a31 a41a12 a22 a32 a42a13 a23 a33 a43 a14 a24 a34 a44

¥ In other words, the element aij of the transpose is equal to the aji element of the original matrix. The transpose has a variety of functions in matrix algebra. One simple advantage is that it allows a column vector to be written as a row. For example, if

{C} 5 μ c1c2 c3 c4

∂ then

{C}T 5 :c1 c2 c3 c4 ; where the superscript T designates the transpose. For example, this can save space when writing a column vector in a manuscript. In addition, the transpose has numerous math- ematical applications. The trace of a matrix is the sum of the elements on its principal diagonal. It is designated as tr [A] and is computed as

tr [A] 5 a n

i51 aii

The trace will be used in our discussion of eigenvalues in Chap. 27. The fi nal matrix manipulation that will have utility in our discussion is augmentation. A matrix is augmented by the addition of a column (or columns) to the original matrix. For example, suppose we have a matrix of coeffi cients:

[A] 5 £ a11 a12 a13a21 a22 a23 a31 a32 a33

§

cha9792x_p03_230-244.indd Page 239 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 239 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

240 LINEAR ALGEBRAIC EQUATIONS

We might wish to augment this matrix [A] with an identity matrix (recall Box PT3.1) to yield a 3-by-6-dimensional matrix:

[A] 5 £ a11 a12 a13a21 a22 a23 a31 a32 a33

   

1 0 0

0 1 0

0 0 1

§ Such an expression has utility when we must perform a set of identical operations on two matrices. Thus, we can perform the operations on the single augmented matrix rather than on the two individual matrices.

PT3.2.3 Representing Linear Algebraic Equations in Matrix Form

It should be clear that matrices provide a concise notation for representing simultaneous linear equations. For example, Eq. (PT3.1) can be expressed as

[A]{X} 5 {B} (PT3.5)

where [A] is the n by n square matrix of coeffi cients,

[A] 5 Fa11 a12 p a1na21 a22 p a2n. . . . . .

. . .

an1 an2 p ann

V {B} is the n by 1 column vector of constants,

{B}T 5 :b1 b2 p bn ; and {X} is the n by 1 column vector of unknowns:

{X}T 5 :x1 x2 p xn ; Recall the defi nition of matrix multiplication [Eq. (PT3.2) or Box PT3.2] to convince yourself that Eqs. (PT3.1) and (PT3.5) are equivalent. Also, realize that Eq. (PT3.5) is a valid matrix multiplication because the number of columns, n, of the fi rst matrix [A] is equal to the number of rows, n, of the second matrix {X}. This part of the book is devoted to solving Eq. (PT3.5) for {X}. A formal way to obtain a solution using matrix algebra is to multiply each side of the equation by the inverse of [A] to yield

[A]21[A]{X} 5 [A]21{B}

Because [A]21[A] equals the identity matrix, the equation becomes

{X} 5 [A]21{B} (PT3.6)

Therefore, the equation has been solved for {X}. This is another example of how the inverse plays a role in matrix algebra that is similar to division. It should be noted that this is not a very effi cient way to solve a system of equations. Thus, other approaches

cha9792x_p03_230-244.indd Page 240 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 240 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.3 ORIENTATION 241

are employed in numerical algorithms. However, as discussed in Chap. 10, the matrix inverse itself has great value in the engineering analyses of such systems. Finally, we will sometimes fi nd it useful to augment [A] with {B}. For example, if n 5 3, this results in a 3-by-4-dimensional matrix:

[A] 5 £ a11 a12 a13  b1a21 a22 a23  b2 a31 a32 a33  b3

§ (PT3.7) Expressing the equations in this form is useful because several of the techniques for solving linear systems perform identical operations on a row of coeffi cients and the cor- responding right-hand-side constant. As expressed in Eq. (PT3.7), we can perform the manipulation once on an individual row of the augmented matrix rather than separately on the coeffi cient matrix and the right-hand-side vector.

PT3.3 ORIENTATION

Before proceeding to the numerical methods, some further orientation might be helpful. The following is intended as an overview of the material discussed in Part Three. In addition, we have formulated some objectives to help focus your efforts when studying the material.

PT3.3.1 Scope and Preview

Figure PT3.5 provides an overview for Part Three. Chapter 9 is devoted to the most fundamental technique for solving linear algebraic systems: Gauss elimination. Before launching into a detailed discussion of this technique, a preliminary section deals with simple methods for solving small systems. These approaches are presented to provide you with visual insight and because one of the methods—the elimination of unknowns— represents the basis for Gauss elimination. After the preliminary material, “naive’’ Gauss elimination is discussed. We start with this “stripped-down” version because it allows the fundamental technique to be elabo- rated on without complicating details. Then, in subsequent sections, we discuss potential problems of the naive approach and present a number of modifi cations to minimize and circumvent these problems. The focus of this discussion will be the process of switching rows, or partial pivoting. Chapter 10 begins by illustrating how Gauss elimination can be formulated as an LU decomposition solution. Such solution techniques are valuable for cases where many right-hand-side vectors need to be evaluated. It is shown how this attribute allows effi cient calculation of the matrix inverse, which has tremendous utility in engineering practice. Finally, the chapter ends with a discussion of matrix condition. The condition number is introduced as a measure of the loss of signifi cant digits of accuracy that can result when solving ill-conditioned matrices. The beginning of Chap. 11 focuses on special types of systems of equations that have broad engineering application. In particular, effi cient techniques for solving tridiagonal systems are presented. Then, the remainder of the chapter focuses on an alternative to elimination methods called the Gauss-Seidel method. This technique is similar in spirit to

cha9792x_p03_230-244.indd Page 241 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 241 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

242 LINEAR ALGEBRAIC EQUATIONS

FIGURE PT3.5 Schematic of the organization of the material in Part Three: Linear Algebraic Equations.

PT 3.1 Motivation

PT 3.2 Mathematical background PT 3.3

Orientation

9.1 Small

systems

9.2 Naive Gauss eliminationPART 3

Linear Algebraic Equations

PT 3.6 Advanced methods

EPILOGUE

CHAPTER 9

Gauss Elimination

PT 3.5 Important formulas

PT 3.4 Trade-offs

12.4 Mechanical engineering

12.3 Electrical

engineering

12.2 Civil

engineering 12.1 Chemical

engineering 11.3

Software

11.2 Gauss- Seidel

11.1 Special

matrices

CHAPTER 10

LU Decomposition and

Matrix Inversion

CHAPTER 11

Special Matrices and Gauss-Seidel

CHAPTER 12

Engineering Case Studies

10.3 System

condition

10.2 Matrix inverse

10.1 LU

decomposition

9.7 Gauss-Jordan

9.6 Nonlinear systems

9.5 Complex systems

9.4 Remedies

9.3 Pitfalls

cha9792x_p03_230-244.indd Page 242 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 242 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.3 ORIENTATION 243

the approximate methods for roots of equations that were discussed in Chap. 6. That is, the technique involves guessing a solution and then iterating to obtain a refi ned estimate. The chapter ends with information related to solving linear algebraic equations with software packages. Chapter 12 demonstrates how the methods can actually be applied for problem solv- ing. As with other parts of the book, applications are drawn from all fi elds of engineering. Finally, an epilogue is included at the end of Part Three. This review includes dis- cussion of trade-offs that are relevant to implementation of the methods in engineering practice. This section also summarizes the important formulas and advanced methods related to linear algebraic equations. As such, it can be used before exams or as a refresher after you have graduated and must return to linear algebraic equations as a professional.

PT3.3.2 Goals and Objectives

Study Objectives. After completing Part Three, you should be able to solve problems involving linear algebraic equations and appreciate the application of these equations in many fi elds of engineering. You should strive to master several techniques and assess their reliability. You should understand the trade-offs involved in selecting the “best” method (or methods) for any particular problem. In addition to these general objectives, the specifi c concepts listed in Table PT3.1 should be assimilated and mastered.

Computer Objectives. Your most fundamental computer objectives are to be able to solve a system of linear algebraic equations and to evaluate the matrix inverse. You will

TABLE PT3.1 Specifi c study objectives for Part Three.

1. Understand the graphical interpretation of ill-conditioned systems and how it relates to the determinant.

2. Be familiar with terminology: forward elimination, back substitution, pivot equation, and pivot coeffi cient.

3. Understand the problems of division by zero, round-off error, and ill-conditioning. 4. Know how to compute the determinant using Gauss elimination. 5. Understand the advantages of pivoting; realize the difference between partial and complete

pivoting. 6. Know the fundamental difference between Gauss elimination and the Gauss-Jordan method and

which is more effi cient. 7. Recognize how Gauss elimination can be formulated as an LU decomposition. 8. Know how to incorporate pivoting and matrix inversion into an LU decomposition algorithm. 9. Know how to interpret the elements of the matrix inverse in evaluating stimulus response

computations in engineering. 10. Realize how to use the inverse and matrix norms to evaluate system condition. 11. Understand how banded and symmetric systems can be decomposed and solved effi ciently. 12. Understand why the Gauss-Seidel method is particularly well suited for large, sparse systems of

equations. 13. Know how to assess diagonal dominance of a system of equations and how it relates to whether

the system can be solved with the Gauss-Seidel method. 14. Understand the rationale behind relaxation; know where underrelaxation and overrelaxation are

appropriate.

cha9792x_p03_230-244.indd Page 243 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 243 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

244 LINEAR ALGEBRAIC EQUATIONS

want to have subprograms developed for LU decomposition of both full and tridiagonal matrices. You may also want to have your own software to implement the Gauss-Seidel method. You should know how to use packages to solve linear algebraic equations and fi nd the matrix inverse. You should become familiar with how the same evaluations can be implemented on popular software packages such as Excel, MATLAB software, and Mathcad.

cha9792x_p03_230-244.indd Page 244 15/10/13 7:20 PM f-479 cha9792x_p03_230-244.indd Page 244 15/10/13 7:20 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9 C H A P T E R 9

245

Gauss Elimination

This chapter deals with simultaneous linear algebraic equations that can be represented generally as

a11x1 1 a12x2 1 p 1 a1nxn 5 b1 a21x1 1 a22x2 1 p 1 a2nxn 5 b2 . . (9.1)

. .

. .

an1x1 1 an2x2 1 p 1 annxn 5 bn where the a’s are constant coeffi cients and the b’s are constants. The technique described in this chapter is called Gauss elimination because it involves combining equations to eliminate unknowns. Although it is one of the earliest methods for solving simultaneous equations, it remains among the most important algorithms in use today and is the basis for linear equation solving on many popular software packages.

9.1 SOLVING SMALL NUMBERS OF EQUATIONS

Before proceeding to the computer methods, we will describe several methods that are appropriate for solving small (n # 3) sets of simultaneous equations and that do not require a computer. These are the graphical method, Cramer’s rule, and the elimination of unknowns.

9.1.1 The Graphical Method

A graphical solution is obtainable for two equations by plotting them on Cartesian co- ordinates with one axis corresponding to x1 and the other to x2. Because we are dealing with linear systems, each equation is a straight line. This can be easily illustrated for the general equations

a11x1 1 a12x2 5 b1 a21x1 1 a22x2 5 b2

cha9792x_ch09_245-277.indd Page 245 15/10/13 9:07 PM f-479 cha9792x_ch09_245-277.indd Page 245 15/10/13 9:07 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

246 GAUSS ELIMINATION

Both equations can be solved for x2:

x2 5 2aa11a12b x1 1 b1 a12

x2 5 2aa21a22b x1 1 b2 a22

Thus, the equations are now in the form of straight lines; that is, x2 5 (slope) x1 1 inter- cept. These lines can be graphed on Cartesian coordinates with x2 as the ordinate and x1 as the abscissa. The values of x1 and x2 at the intersection of the lines represent the solution.

EXAMPLE 9.1 The Graphical Method for Two Equations

Problem Statement. Use the graphical method to solve

3x1 1 2x2 5 18 (E9.1.1)

2x1 1 2x2 5 2 (E9.1.2)

Solution. Let x1 be the abscissa. Solve Eq. (E9.1.1) for x2:

x2 5 2 3

2 x1 1 9

which, when plotted on Fig. 9.1, is a straight line with an intercept of 9 and a slope of 23y2.

FIGURE 9.1 Graphical solution of a set of two simultaneous linear algebraic equations. The intersection of the lines represents the solution.

0 62 4 0

6

2

4

8

x2

x1

Solution: x1 � 4; x2 � 3

�x 1 �

2x 2 �

2

3x 1 �

2x 2 �

18

cha9792x_ch09_245-277.indd Page 246 15/10/13 9:07 PM f-479 cha9792x_ch09_245-277.indd Page 246 15/10/13 9:07 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.1 SOLVING SMALL NUMBERS OF EQUATIONS 247

For three simultaneous equations, each equation would be represented by a plane in a three-dimensional coordinate system. The point where the three planes intersect would represent the solution. Beyond three equations, graphical methods break down and, con- sequently, have little practical value for solving simultaneous equations. However, they sometimes prove useful in visualizing properties of the solutions. For example, Fig. 9.2 depicts three cases that can pose problems when solving sets of linear equations. Figure 9.2a shows the case where the two equations represent parallel lines. For such situations, there is no solution because the lines never cross. Figure 9.2b depicts the case where the two lines are coincident. For such situations there is an infi nite number of solutions. Both types of systems are said to be singular. In addition, systems that are very close to being singular (Fig. 9.2c) can also cause problems. These systems are said to be ill-conditioned. Graphically, this corresponds to the fact that it is diffi cult to identify the exact point at which the lines intersect. Ill-conditioned systems will also pose problems when they are encountered during the numerical solution of linear equations. This is because they will be extremely sensitive to round-off error (recall Sec. 4.2.3).

Equation (E9.1.2) can also be solved for x2:

x2 5 1

2 x1 1 1

which is also plotted on Fig. 9.1. The solution is the intersection of the two lines at x1 5 4 and x2 5 3. This result can be checked by substituting these values into the original equations to yield

3(4) 1 2(3) 5 18

2(4) 1 2(3) 5 2

Thus, the results are equivalent to the right-hand sides of the original equations.

FIGURE 9.2 Graphical depiction of singular and ill-conditioned systems: (a) no solution, (b) infi nite solutions, and (c) ill-conditioned system where the slopes are so close that the point of intersection is diffi cult to detect visually.

x2

x1

x 1 � x 2 �

1

x 1 � x 2 �

(a) (b)

x2

x1

�x 1 � 2

x 2 � 2 x 1

� x 2 � 1

(c)

x2

x1

x1 � x2 �

1

� 2 1

x1 � x2 �

1.1

� 5 2.3� 2

1

� 2 1

2 1

� 2 1

cha9792x_ch09_245-277.indd Page 247 15/10/13 9:07 PM f-479 cha9792x_ch09_245-277.indd Page 247 15/10/13 9:07 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

248 GAUSS ELIMINATION

9.1.2 Determinants and Cramer’s Rule

Cramer’s rule is another solution technique that is best suited to small numbers of equa- tions. Before describing this method, we will briefl y introduce the concept of the deter- minant, which is used to implement Cramer’s rule. In addition, the determinant has relevance to the evaluation of the ill-conditioning of a matrix.

Determinants. The determinant can be illustrated for a set of three equations:

[A]{X} 5 {B}

where [A] is the coeffi cient matrix:

[A] 5 £ a11 a12 a13 a21 a22 a23 a31 a32 a33

§

The determinant D of this system is formed from the coeffi cients of the equation, as in

D 5 † a11 a12 a13 a21 a22 a23 a31 a32 a33

† (9.2)

Although the determinant D and the coeffi cient matrix [A] are composed of the same elements, they are completely different mathematical concepts. That is why they are distinguished visually by using brackets to enclose the matrix and straight lines to enclose the determinant. In contrast to a matrix, the determinant is a single number. For example, the value of the second-order determinant

D 5 ` a11 a12 a21 a22

` is calculated by

D 5 a11a22 2 a12a21 (9.3)

For the third-order case [Eq. (9.2)], a single numerical value for the determinant can be computed as

D 5 a11 ` a22 a23a32 a33 ` 2a12 ` a21 a23 a31 a33

` 1a13 ` a21 a22a31 a32 ` (9.4) where the 2 by 2 determinants are called minors.

EXAMPLE 9.2 Determinants

Problem Statement. Compute values for the determinants of the systems represented in Figs. 9.1 and 9.2.

Solution. For Fig. 9.1:

D 5 ` 3 2 21 2

` 5 3(2) 2 2(21) 5 8

cha9792x_ch09_245-277.indd Page 248 15/10/13 9:07 PM f-479 cha9792x_ch09_245-277.indd Page 248 15/10/13 9:07 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.1 SOLVING SMALL NUMBERS OF EQUATIONS 249

In the foregoing example, the singular systems had zero determinants. Additionally, the results suggest that the system that is almost singular (Fig. 9.2c) has a determinant that is close to zero. These ideas will be pursued further in our subsequent discussion of ill-conditioning (Sec. 9.3.3).

Cramer’s Rule. This rule states that each unknown in a system of linear algebraic equa- tions may be expressed as a fraction of two determinants with denominator D and with the numerator obtained from D by replacing the column of coeffi cients of the unknown in question by the constants b1, b2, . . . , bn. For example, x1 would be computed as

x1 5

† b1 a12 a13 b2 a22 a23 b3 a32 a33

† D

(9.5)

EXAMPLE 9.3 Cramer’s Rule

Problem Statement. Use Cramer’s rule to solve

0.3x1 1 0.52x2 1 x3 5 20.01 0.5x1 1 x2 1 1.9x3 5 0.67 0.1x1 1 0.3x2 1 0.5x3 5 20.44

Solution. The determinant D can be written as [Eq. (9.2)]

D 5 † 0.3 0.52 1

0.5 1 1.9

0.1 0.3 0.5

The minors are [Eq. (9.3)]

A1 5 ` 1 1.90.3 0.5 ` 5 1(0.5) 2 1.9(0.3) 5 20.07

A2 5 ` 0.5 1.90.1 0.5 ` 5 0.5(0.5) 2 1.9(0.1) 5 0.06

For Fig. 9.2a:

D 5 `21y2 1 21y2 1

` 5 21 2

(1) 2 1 a21 2 b 5 0

For Fig. 9.2b:

D 5 `21y2 1 21 2

` 5 21 2

(2) 2 1(21) 5 0

For Fig. 9.2c:

D 5 ` 21y2 1 22.3y5 1

` 5 21 2

(1) 2 1 a22.3 5 b 5 20.04

cha9792x_ch09_245-277.indd Page 249 15/10/13 9:07 PM f-479 cha9792x_ch09_245-277.indd Page 249 15/10/13 9:07 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

250 GAUSS ELIMINATION

A3 5 ` 0.5 10.1 0.3 ` 5 0.5(0.3) 2 1(0.1) 5 0.05 These can be used to evaluate the determinant, as in [Eq. (9.4)]

D 5 0.3(20.07) 2 0.52(0.06) 1 1(0.05) 5 20.0022

Applying Eq. (9.5), the solution is

x1 5

† 20.01 0.52 1

 0.67 1 1.9

20.44 0.3 0.5

† 20.0022

5 0.03278

20.0022 5 214.9

x2 5

† 0.3 20.01 1

0.5  0.67 1.9

0.1 20.44 0.5

† 20.0022

5 0.0649

20.0022 5 229.5

x3 5

† 0.3 0.52 20.01

0.5 1  0.67

0.1 0.3 20.44

† 20.0022

5 20.04356

20.0022 5 19.8

For more than three equations, Cramer’s rule becomes impractical because, as the number of equations increases, the determinants are time consuming to evaluate by hand (or by computer). Consequently, more effi cient alternatives are used. Some of these al- ternatives are based on the last noncomputer solution technique covered in the next section—the elimination of unknowns.

9.1.3 The Elimination of Unknowns

The elimination of unknowns by combining equations is an algebraic approach that can be illustrated for a set of two equations:

a11x1 1 a12x2 5 b1 (9.6)

a21x1 1 a22x2 5 b2 (9.7)

The basic strategy is to multiply the equations by constants so that one of the unknowns will be eliminated when the two equations are combined. The result is a single equation that can be solved for the remaining unknown. This value can then be substituted into either of the original equations to compute the other variable. For example, Eq. (9.6) might be multiplied by a21 and Eq. (9.7) by a11 to give

a11a21x1 1 a12a21x2 5 b1a21 (9.8)

a21a11x1 1 a22a11x2 5 b2a11 (9.9)

cha9792x_ch09_245-277.indd Page 250 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 250 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.1 SOLVING SMALL NUMBERS OF EQUATIONS 251

Subtracting Eq. (9.8) from Eq. (9.9) will, therefore, eliminate the x1 term from the equa- tions to yield

a22a11x2 2 a12a21x2 5 b2a11 2 b1a21

which can be solved for

x2 5 a11b2 2 a21b1

a11a22 2 a12a21 (9.10)

Equation (9.10) can then be substituted into Eq. (9.6), which can be solved for

x1 5 a22b1 2 a12b2

a11a22 2 a12a21 (9.11)

Notice that Eqs. (9.10) and (9.11) follow directly from Cramer’s rule, which states

x1 5

` b1 a12 b2 a22

` ` a11 a12 a21 a22

` 5

b1a22 2 a12b2 a11a22 2 a12a21

x2 5

` a11 b1 a21 b2

` ` a11 a12 a21 a22

` 5

a11b22 2 b1a21 a11a22 2 a12a21

EXAMPLE 9.4 Elimination of Unknowns

Problem Statement. Use the elimination of unknowns to solve (recall Example 9.1)

3x1 1 2x2 5 18

2x1 1 2x2 5 2

Solution. Using Eqs. (9.11) and (9.10),

x1 5 2(18) 2 2(2)

3(2) 2 2(21) 5 4

x2 5 3(2) 2 (21)18

3(2) 2 2(21) 5 3

which is consistent with our graphical solution (Fig. 9.1).

The elimination of unknowns can be extended to systems with more than two or three equations. However, the numerous calculations that are required for larger systems make the method extremely tedious to implement by hand. However, as described in the next section, the technique can be formalized and readily programmed for the computer.

cha9792x_ch09_245-277.indd Page 251 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 251 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

252 GAUSS ELIMINATION

9.2 NAIVE GAUSS ELIMINATION

In the previous section, the elimination of unknowns was used to solve a pair of simul- taneous equations. The procedure consisted of two steps:

1. The equations were manipulated to eliminate one of the unknowns from the equations. The result of this elimination step was that we had one equation with one unknown.

2. Consequently, this equation could be solved directly and the result back-substituted into one of the original equations to solve for the remaining unknown.

This basic approach can be extended to large sets of equations by developing a systematic scheme or algorithm to eliminate unknowns and to back-substitute. Gauss elimination is the most basic of these schemes. This section includes the systematic techniques for forward elimination and back sub- stitution that comprise Gauss elimination. Although these techniques are ideally suited for implementation on computers, some modifi cations will be required to obtain a reliable algo- rithm. In particular, the computer program must avoid division by zero. The following method is called “naive” Gauss elimination because it does not avoid this problem. Subsequent sections will deal with the additional features required for an effective computer program. The approach is designed to solve a general set of n equations:

a11x1 1 a12x2 1 a13x3 1 p 1 a1nxn 5 b1 (9.12a) a21x1 1 a22x2 1 a23x3 1 p 1 a2nxn 5 b2 (9.12b) . . . . . .

an1x1 1 an2x2 1 an3x3 1 p 1 annxn 5 bn (9.12c)

As was the case with the solution of two equations, the technique for n equations consists of two phases: elimination of unknowns and solution through back substitution.

Forward Elimination of Unknowns. The fi rst phase is designed to reduce the set of equations to an upper triangular system (Fig. 9.3). The initial step will be to eliminate the fi rst unknown, x1, from the second through the nth equations. To do this, multiply Eq. (9.12a) by a21Ya11 to give

a21x1 1 a21 a11

a12x2 1 p 1 a21 a11

a1nxn 5 a21 a11

b1 (9.13)

Now, this equation can be subtracted from Eq. (9.12b) to give

aa22 2 a21a11 a12b x2 1 p 1 aa2n 2 a21 a11

a1nb

xn 5 b2 2 a21 a11

b1

or

a¿22x2 1 p 1 a¿2nxn 5 b¿2 where the prime indicates that the elements have been changed from their original values. The procedure is then repeated for the remaining equations. For instance, Eq. (9.12a) can be multiplied by a31ya11 and the result subtracted from the third equation. Repeating

cha9792x_ch09_245-277.indd Page 252 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 252 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.2 NAIVE GAUSS ELIMINATION 253

the procedure for the remaining equations results in the following modifi ed system:

a11x1 1 a12x2 1 a13x3 1 p 1 a1nxn 5 b1 (9.14a) a¿22x2 1 a¿23x3 1 p 1 a¿2nxn 5 b¿2 (9.14b) a¿32x2 1 a¿33x3 1 p 1 a¿3nxn 5 b¿3 (9.14c)

. .

. .

. .

a¿n2x2 1 a¿n3x3 1 p 1 a¿nnxn 5 b¿n (9.14d)

For the foregoing steps, Eq. (9.12a) is called the pivot equation and a11 is called the pivot coeffi cient or element. Note that the process of multiplying the fi rst row by a21ya11 is equivalent to dividing it by a11 and multiplying it by a21. Sometimes the division operation is referred to as normalization. We make this distinction because a zero pivot element can interfere with normalization by causing a division by zero. We will return to this important issue after we complete our description of naive Gauss elimination. Now repeat the above to eliminate the second unknown from Eq. (9.14c) through (9.14d). To do this multiply Eq. (9.14b) by a932ya922 and subtract the result from Eq. (9.14c). Perform a similar elimination for the remaining equations to yield

a11x1 1 a12x2 1 a13x3 1 p 1 a1nxn 5 b1 a¿22x2 1 a¿23x3 1 p 1 a¿2nxn 5 b¿2

a–33x3 1 p 1 a–3nxn 5 b–2 . .

. .

. .

a–n3x3 1 p 1 a–nnxn 5 b–n where the double prime indicates that the elements have been modifi ed twice.

FIGURE 9.3 The two phases of Gauss elimination: forward elimination and back substitution. The primes indicate the number of times that the coeffi cients and constants have been modifi ed.

£ a11 a12 a13 b1 a21 a22 a23 b2 a31 a32 a33 b3

§

2

£ a11 a12 a13 b1

a'22 a'23 b'2 a''33 b''3

§ 2 x3 5 b''3ya''33 x2 5 (b'2 2 a'2333)ya'22 x1 5 (b1 2 a1232 2 a1333)ya11

Forward elimination

Back substitution

cha9792x_ch09_245-277.indd Page 253 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 253 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

254 GAUSS ELIMINATION

The procedure can be continued using the remaining pivot equations. The fi nal ma- nipulation in the sequence is to use the (n 2 1)th equation to eliminate the xn21 term from the nth equation. At this point, the system will have been transformed to an upper triangular system (recall Box PT3.1):

a11x1 1 a12x2 1 a13x3 1 p 1 a1nxn 5 b1 (9.15a) a¿22x2 1 a¿23x3 1 p 1 a¿2nxn 5 b¿2 (9.15b)

a–33x3 1 p 1 a–3nxn 5 b–3 (9.15c) . .

. .

. .

a(n21)nn xn 5 bn (n21) (9.15d)

Pseudocode to implement forward elimination is presented in Fig. 9.4a. Notice that three nested loops provide a concise representation of the process. The outer loop moves down the matrix from one pivot row to the next. The middle loop moves below the pivot row to each of the subsequent rows where elimination is to take place. Finally, the innermost loop pro- gresses across the columns to eliminate or transform the elements of a particular row.

Back Substitution. Equation (9.15d) can now be solved for xn:

xn 5 b(n21)n a(n21)nn

(9.16)

This result can be back-substituted into the (n 2 l)th equation to solve for xn21. The procedure, which is repeated to evaluate the remaining x’s, can be represented by the following formula:

xi 5

b(i21)i 2 a n

j5i11 a(i21)ij xj

a(i21)ii   for i 5 n 2 1, n 2 2, p , 1 (9.17)

(a) DOFOR k 5 1, n 2 1 DOFOR i 5 k 1 1, n factor 5 ai,k y ak,k DOFOR j 5 k 1 1 to n ai,j 5 ai,j 2 factor ? ak,j END DO bi 5 bi 2 factor ? bk END DO END DO (b) xn 5 bn y an,n DOFOR i 5 n 2 1, 1, 21 sum 5 bi DOFOR j 5 i 1 1, n sum 5 sum 2 ai,j ? xj END DO xi 5 sum y ai,i END DO

FIGURE 9.4 Pseudocode to perform (a) for- ward elimination and (b) back substitution.

cha9792x_ch09_245-277.indd Page 254 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 254 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.2 NAIVE GAUSS ELIMINATION 255

Pseudocode to implement Eqs. (9.16) and (9.17) is presented in Fig. 9.4b. Notice the similarity between this pseudocode and that in Fig. PT3.4 for matrix multiplication. As with Fig. PT3.4, a temporary variable, sum, is used to accumulate the summation from Eq. (9.17). This results in a somewhat faster execution time than if the summation were accumulated in bi. More importantly, it allows effi cient improvement in precision if the variable, sum, is declared in double precision.

EXAMPLE 9.5 Naive Gauss Elimination

Problem Statement. Use Gauss elimination to solve

3x1 2 0.1x2 2 0.2x3 5 7.85 (E9.5.1)

0.1x1 1 7x2 2 0.3x3 5 219.3 (E9.5.2)

0.3x1 2 0.2x2 1 10x3 5 71.4 (E9.5.3)

Carry six signifi cant fi gures during the computation.

Solution. The fi rst part of the procedure is forward elimination. Multiply Eq. (E9.5.1) by (0.1)y3 and subtract the result from Eq. (E9.5.2) to give

7.00333x2 2 0.293333x3 5 219.5617

Then multiply Eq. (E9.5.1) by (0.3)y3 and subtract it from Eq. (E9.5.3) to eliminate x1. After these operations, the set of equations is

3x1    20.1x2     20.2x3 5 7.85 (E9.5.4)

7.00333x2 2 0.293333x3 5 219.5617 (E9.5.5)

20.190000x2 1 10.0200x3 5 70.6150 (E9.5.6)

To complete the forward elimination, x2 must be removed from Eq. (E9.5.6). To accom- plish this, multiply Eq. (E9.5.5) by 20.190000y7.00333 and subtract the result from Eq. (E9.5.6). This eliminates x2 from the third equation and reduces the system to an upper triangular form, as in

3x1    20.1x2      20.2x3 5 7.85 (E9.5.7)

7.00333x2 2 0.293333x3 5 219.5617 (E9.5.8)

10.0120x3 5 70.0843 (E9.5.9)

We can now solve these equations by back substitution. First, Eq. (E9.5.9) can be solved for

x3 5 70.0843

10.0120 5 7.0000 (E9.5.10)

This result can be back-substituted into Eq. (E9.5.8):

7.00333x2 2 0.293333(7.0000) 5 219.5617

which can be solved for

x2 5 2 19.5617 1 0.293333(7.0000)

7.00333 5 22.50000 (E9.5.11)

cha9792x_ch09_245-277.indd Page 255 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 255 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

256 GAUSS ELIMINATION

Finally, Eqs. (E9.5.10) and (E9.5.11) can be substituted into Eq. (E9.5.4):

3x1 2 0.1(22.50000) 2 0.2(7.0000) 5 7.85

which can be solved for

x1 5 7.85 1 0.1(22.50000) 1 0.2(7.0000)

3 5 3.00000

The results are identical to the exact solution of x1 5 3, x2 5 22.5, and x3 5 7. This can be verifi ed by substituting the results into the original equation set

3(3) 2 0.1(22.5) 2 0.2(7) 5 7.85

0.1(3) 1 7(22.5) 2 0.3(7) 5 219.3

0.3(3) 2 0.2(22.5) 1 10(7) 5 71.4

9.2.1 Operation Counting

The execution time of Gauss elimination depends on the amount of fl oating-point operations (or fl ops) involved in the algorithm. On modern computers using math copro- cessors, the time consumed to perform addition/subtraction and multiplication/division is about the same. Therefore, totaling up these operations provides insight into which parts of the algorithm are most time consuming and how computation time increases as the system gets larger. Before analyzing naive Gauss elimination, we will fi rst defi ne some quantities that facilitate operation counting:

a m

i51 cf (i) 5 ca

m

i51 f(i)  a

m

i51 f(i) 1 g(i) 5 a

m

i51 f(i) 1 a

m

i51 g(i) (9.18a,b)

a m

i51 1 5 1 1 1 1 1 1 p 1 1 5 m a

m

i5k 1 5 m 2 k 1 1 (9.18c,d)

a m

i51 i 5 1 1 2 1 3 1 p 1 m 5

m(m 1 1)

2 5

m2

2 1 O(m) (9.18e)

a m

i51 i2 5 12 1 22 1 32 1 p 1 m2 5

m(m 1 1)(2m 1 1)

6 5

m3

3 1 O(m2) (9.18f )

where O(mn) means “terms of order mn and lower.” Now let us examine the naive Gauss elimination algorithm (Fig. 9.4a) in detail. We will fi rst count the fl ops in the elimination stage. On the fi rst pass through the outer loop, k 5 1. Therefore, the limits on the middle loop are from i 5 2 to n. According to Eq. (9.18d), this means that the number of iterations of the middle loop will be

a n

i52 1 5 n 2 2 1 1 5 n 2 1 (9.19)

For every one of these iterations, there is one division to defi ne the factor. The interior loop then performs a single multiplication and subtraction for each iteration from j 5 2 to n. Finally, there is one additional multiplication and subtraction for the right-hand-side value.

cha9792x_ch09_245-277.indd Page 256 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 256 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.2 NAIVE GAUSS ELIMINATION 257

Thus, for every iteration of the middle loop, the number of multiplications is

1 1 [n 2 2 1 1] 1 1 5 1 1 n (9.20)

The total multiplications for the fi rst pass through the outer loop is therefore obtained by multiplying Eq. (9.19) by (9.20) to give [n 2 1](1 1 n). In like fashion, the number of subtractions is computed as [n 2 1](n). Similar reasoning can be used to estimate the fl ops for the subsequent iterations of the outer loop. These can be summarized as

Outer Loop Middle Loop Addition/Subtraction Multiplication/Division k i fl ops fl ops

1 2, n (n 2 1)(n) (n 2 1)(n 1 1) 2 3, n (n 2 2)(n – 1) (n 2 2)(n) . . . . . . k k 1 1, n (n 2 k)(n 1 1 2 k) (n 2 k)(n 1 2 2 k) . . . . . .

n 2 1 n, n (1)(2) (1) (3)

Therefore, the total addition/subtraction fl ops for elimination can be computed as

a n21

k51 (n 2 k)(n 1 1 2 k) 5 a

n21

k51 [n(n 1 1) 2 k(2n 1 1) 1 k2]

or

n(n 1 1)a n21

k51 1 2 (2n 1 1)a

n21

k51 k 1 a

n21

k51 k2

Applying some of the relationships from Eq. (9.18) yields

[n3 1 O(n)] 2 [n3 1 O(n2) ] 1 c 1 3

n3 1 O(n2) d 5 n3 3

1 O(n) (9.21)

A similar analysis for the multiplication/division fl ops yields

[n3 1 O(n2) ] 2 [n3 1 O(n)] 1 c 1 3

n3 1 O(n2) d 5 n3 3

1 O(n2) (9.22)

Summing these results gives

2n3

3 1 O(n2)

Thus, the total number of fl ops is equal to 2n3y3 plus an additional component proportional to terms of order n2 and lower. The result is written in this way because as n gets large, the O(n2) and lower terms become negligible. We are therefore justifi ed in concluding that for large n, the effort involved in forward elimination converges on 2n3/3. Because only a single loop is used, back substitution is much simpler to evaluate. The number of addition/subtraction fl ops is equal to n(n 2 1)y2. Because of the extra

cha9792x_ch09_245-277.indd Page 257 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 257 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

258 GAUSS ELIMINATION

division prior to the loop, the number of multiplication/division fl ops is n(n 1 1)y2. These can be added to arrive at a total of

n2 1 O(n)

Thus, the total effort in naive Gauss elimination can be represented as

2n3

3 1 O(n2) 1 n2 1 O(n) ————Sas n increases

2n3

3 1 O(n2) (9.23)

Forward Backward elimination substitution

Two useful general conclusions can be drawn from this analysis:

1. As the system gets larger, the computation time increases greatly. As in Table 9.1, the amount of fl ops increases nearly three orders of magnitude for every order of magnitude increase in the dimension.

2. Most of the effort is incurred in the elimination step. Thus, efforts to make the method more effi cient should probably focus on this step.

9.3 PITFALLS OF ELIMINATION METHODS

Whereas there are many systems of equations that can be solved with naive Gauss elimina- tion, there are some pitfalls that must be explored before writing a general computer program to implement the method. Although the following material relates directly to naive Gauss elimination, the information is relevant for other elimination techniques as well.

9.3.1 Division by Zero

The primary reason that the foregoing technique is called “naive” is that during both the elimination and the back-substitution phases, it is possible that a division by zero can occur. For example, if we use naive Gauss elimination to solve

2x2 1 3x3 5 8

4x1 1 6x2 1 7x3 5 23

2x1 1 x2 1 6x3 5 5

the normalization of the fi rst row would involve division by a11 5 0. Problems also can arise when a coeffi cient is very close to zero. The technique of pivoting has been devel- oped to partially avoid these problems. It will be described in Sec. 9.4.2.

TABLE 9.1 Number of Flops for Gauss Elimination.

Back Total Percent Due n Elimination Substitution Flops 2n3/3 to Elimination

10 705 100 805 667 87.58% 100 671550 10000 681550 666667 98.53% 1000 6.67 3 108 1 3 106 6.68 3 108 6.67 3 108 99.85%

cha9792x_ch09_245-277.indd Page 258 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 258 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.3 PITFALLS OF ELIMINATION METHODS 259

9.3.2 Round-Off Errors

Even though the solution in Example 9.5 was close to the true answer, there was a slight discrepancy in the result for x3 [Eq. (E9.5.10)]. This discrepancy, which amounted to a relative error of 20.00043 percent, was due to our use of six signifi cant fi gures during the computation. If we had used more signifi cant fi gures, the error in the results would be reduced further. If we had used fractions instead of decimals (and consequently avoided round-off altogether), the answers would have been exact. However, because computers carry only a limited number of signifi cant fi gures (recall Sec. 3.4.1), round-off errors can occur and must be considered when evaluating the results. The problem of round-off error can become particularly important when large num- bers of equations are to be solved. This is due to the fact that every result is dependent on previous results. Consequently, an error in the early steps will tend to propagate—that is, it will cause errors in subsequent steps. Specifying the system size where round-off error becomes signifi cant is complicated by the fact that the type of computer and the properties of the equations are determining factors. A rough rule of thumb is that round-off error may be important when dealing with 100 or more equations. In any event, you should always substitute your answers back into the original equations to check whether a substantial error has occurred. How- ever, as discussed below, the magnitudes of the coeffi cients themselves can infl uence whether such an error check ensures a reliable result.

9.3.3 Ill-Conditioned Systems

The adequacy of the solution depends on the condition of the system. In Sec. 9.1.1, a graph- ical depiction of system condition was developed. As discussed in Sec. 4.2.3, well-conditioned systems are those where a small change in one or more of the coeffi cients results in a simi- lar small change in the solution. Ill-conditioned systems are those where small changes in coeffi cients result in large changes in the solution. An alternative interpretation of ill-condi- tioning is that a wide range of answers can approximately satisfy the equations. Because round-off errors can induce small changes in the coeffi cients, these artifi cial changes can lead to large solution errors for ill-conditioned systems, as illustrated in the following example.

EXAMPLE 9.6 Ill-Conditioned Systems

Problem Statement. Solve the following system:

x1 1 2x2 5 10 (E9.6.1)

1.1x1 1 2x2 5 10.4 (E9.6.2)

Then, solve it again, but with the coeffi cient of x1 in the second equation modifi ed slightly to 1.05.

Solution. Using Eqs. (9.10) and (9.11), the solution is

x1 5 2(10) 2 2(10.4)

1(2) 2 2(1.1) 5 4

x2 5 1(10.4) 2 1.1(10)

1(2) 2 2(1.1) 5 3

cha9792x_ch09_245-277.indd Page 259 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 259 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

260 GAUSS ELIMINATION

However, with the slight change of the coeffi cient a21 from 1.1 to 1.05, the result is changed dramatically to

x1 5 2(10) 2 2(10.4)

1(2) 2 2(1.05) 5 8

x2 5 1(10.4) 2 1.1(10)

1(2) 2 2(1.05) 5 1

Notice that the primary reason for the discrepancy between the two results is that the denominator represents the difference of two almost-equal numbers. As illustrated previously in Sec. 3.4.2, such differences are highly sensitive to slight variations in the numbers being manipulated. At this point, you might suggest that substitution of the results into the original equations would alert you to the problem. Unfortunately, for ill-conditioned systems this is often not the case. Substitution of the erroneous values of x1 5 8 and x2 5 1 into Eqs. (E9.6.1) and (E9.6.2) yields

8 1 2(1) 5 10 5 10

1.1(8) 1 2(1) 5 10.8 > 10.4

Therefore, although x1 5 8 and x2 5 1 is not the true solution to the original problem, the error check is close enough to possibly mislead you into believing that your solutions are adequate.

As was done previously in the section on graphical methods, a visual representative of ill-conditioning can be developed by plotting Eqs. (E9.6.1) and (E9.6.2) (recall Fig. 9.2). Because the slopes of the lines are almost equal, it is visually diffi cult to see exactly where they intersect. This visual diffi culty is refl ected quantitatively in the nebulous results of Example 9.6. We can mathematically characterize this situation by writing the two equa- tions in general form:

a11x1 1 a12x2 5 b1 (9.24)

a21x1 1 a22x2 5 b2 (9.25)

Dividing Eq. (9.24) by a12 and Eq. (9.25) by a22 and rearranging yields alternative ver- sions that are in the format of straight lines [x2 5 (slope) x1 1 intercept]:

x2 5 2 a11 a12

x1 1 b1 a12

x2 5 2 a21 a22

x1 1 b2 a22

Consequently, if the slopes are nearly equal,

a11 a12

> a21 a22

cha9792x_ch09_245-277.indd Page 260 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 260 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.3 PITFALLS OF ELIMINATION METHODS 261

or, cross-multiplying,

a11a22 > a12a21

which can be also expressed as

a11a22 2 a12a21 > 0 (9.26)

Now, recalling that a11a22 2 a12a2l is the determinant of a two-dimensional system [Eq. (9.3)], we arrive at the general conclusion that an ill-conditioned system is one with a determinant close to zero. In fact, if the determinant is exactly zero, the two slopes are identical, which connotes either no solution or an infi nite number of solutions, as is the case for the singular systems depicted in Fig. 9.2a and b. It is diffi cult to specify how close to zero the determinant must be to indicate ill- conditioning. This is complicated by the fact that the determinant can be changed by multiplying one or more of the equations by a scale factor without changing the solution. Consequently, the determinant is a relative value that is infl uenced by the magnitude of the coeffi cients.

EXAMPLE 9.7 Effect of Scale on the Determinant

Problem Statement. Evaluate the determinant of the following systems:

(a) From Example 9.1:

3x1 1 2x2 5 18 (E9.7.1)

2x1 1 2x2 5 2 (E9.7.2)

(b) From Example 9.6:

x1 1 2x2 5 10 (E9.7.3)

1.1x1 1 2x2 5 10.4 (E9.7.4)

(c) Repeat (b) but with the equations multiplied by 10.

Solution.

(a) The determinant of Eqs. (E9.7.1) and (E9.7.2), which are well-conditioned, is

D 5 3(2) 2 2(21) 5 8

(b) The determinant of Eqs. (E9.7.3) and (E9.7.4), which are ill-conditioned, is

D 5 1(2) 2 2(1.1) 5 20.2

(c) The results of (a) and (b) seem to bear out the contention that ill-conditioned systems have near-zero determinants. However, suppose that the ill-conditioned system in (b) is multiplied by 10 to give

10x1 1 20x2 5 100

11x1 1 20x2 5 104

The multiplication of an equation by a constant has no effect on its solution. In ad- dition, it is still ill-conditioned. This can be verifi ed by the fact that multiplying by

cha9792x_ch09_245-277.indd Page 261 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 261 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

262 GAUSS ELIMINATION

As illustrated by the previous example, the magnitude of the coeffi cients interjects a scale effect that complicates the relationship between system condition and determinant size. One way to partially circumvent this diffi culty is to scale the equations so that the maximum element in any row is equal to 1.

EXAMPLE 9.8 Scaling

Problem Statement. Scale the systems of equations in Example 9.7 to a maximum value of 1 and recompute their determinants.

Solution.

(a) For the well-conditioned system, scaling results in

x1 1 0.667x2 5 6

20.5x1 1    x2 5 1

for which the determinant is

D 5 1(1) 2 0.667(20.5) 5 1.333

(b) For the ill-conditioned system, scaling gives

0.5x1 1 x2 5 5

0.55x1 1 x2 5 5.2

for which the determinant is

D 5 0.5(1) 2 1(0.55) 5 20.05

(c) For the last case, scaling changes the system to the same form as in (b) and the determinant is also 20.05. Thus, the scale effect is removed.

a constant has no effect on the graphical solution. However, the determinant is dramatically affected:

D 5 10(20) 2 20(11) 5 220

Not only has it been raised two orders of magnitude, but it is now over twice as large as the determinant of the well-conditioned system in (a).

In a previous section (Sec. 9.1.2), we suggested that the determinant is diffi cult to compute for more than three simultaneous equations. Therefore, it might seem that it does not provide a practical means for evaluating system condition. However, as de- scribed in Box 9.1, there is a simple algorithm that results from Gauss elimination that can be used to evaluate the determinant. Aside from the approach used in the previous example, there are a variety of other ways to evaluate system condition. For example, there are alternative methods for nor- malizing the elements (see Stark, 1970). In addition, as described in the next chapter (Sec. 10.3), the matrix inverse and matrix norms can be employed to evaluate system condition. Finally, a simple (but time-consuming) test is to modify the coeffi cients

cha9792x_ch09_245-277.indd Page 262 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 262 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.3 PITFALLS OF ELIMINATION METHODS 263

slightly and repeat the solution. If such modifi cations lead to drastically different results, the system is likely to be ill-conditioned. As you might gather from the foregoing discussion, ill-conditioned systems are prob- lematic. Fortunately, most linear algebraic equations derived from engineering-problem settings are naturally well-conditioned. In addition, some of the techniques outlined in Sec. 9.4 help to alleviate the problem.

9.3.4 Singular Systems

In the previous section, we learned that one way in which a system of equations can be ill-conditioned is when two or more of the equations are nearly identical. Obviously, it is even worse when the two are identical. In such cases, we would lose one degree of freedom, and would be dealing with the impossible case of n 2 1 equations with n unknowns. Such cases might not be obvious to you, particularly when dealing with large equation sets. Consequently, it would be nice to have some way of automatically detecting singularity. The answer to this problem is neatly offered by the fact that the determinant of a singular system is zero. This idea can, in turn, be connected to Gauss elimination by recognizing that after the elimination step, the determinant can be evaluated as the prod- uct of the diagonal elements (recall Box 9.1). Thus, a computer algorithm can test to discern whether a zero diagonal element is created during the elimination stage. If one is discovered, the calculation can be immediately terminated and a message displayed

Box 9.1 Determinant Evaluation Using Gauss Elimination

In Sec. 9.1.2, we stated that determinant evaluation by expansion of minors was impractical for large sets of equations. Thus, we con- cluded that Cramer’s rule would be applicable only to small sys- tems. However, as mentioned in Sec. 9.3.3, the determinant has value in assessing system condition. It would, therefore, be useful to have a practical method for computing this quantity. Fortunately, Gauss elimination provides a simple way to do this. The method is based on the fact that the determinant of a tri- angular matrix can be simply computed as the product of its diago- nal elements:

D 5 a11a22a33 p ann (B9.1.1)

The validity of this formulation can be illustrated for a 3 by 3 system:

D 5 † a11 a12 a13 0 a22 a23 0 0 a33

where the determinant can be evaluated as [recall Eq. (9.4)]

D 5 a11 ` a22 a230 a33 ` 2a12 ` 0 a23 0 a33

` 1a13 `0 a220 0 ` or, by evaluating the minors (that is, the 2 by 2 determinants),

D 5 a11a22a33 2 a12(0) 1 a13(0) 5 a11a12a33

Recall that the forward-elimination step of Gauss elimination results in an upper triangular system. Because the value of the de- terminant is not changed by the forward-elimination process, the determinant can be simply evaluated at the end of this step via

D 5 a11a¿22 a–33 p a(n21)nn (B9.1.2)

where the superscripts signify the number of times that the ele- ments have been modifi ed by the elimination process. Thus, we can capitalize on the effort that has already been expended in reducing the system to triangular form and, in the bargain, come up with a simple estimate of the determinant. There is a slight modifi cation to the above approach when the program employs partial pivoting (Sec. 9.4.2). For this case, the determinant changes sign every time a row is pivoted. One way to represent this is to modify Eq. (B9.1.2):

D 5 a11a¿22 a–33 p a(n21)nn (21)p (B9.1.3)

where p represents the number of times that rows are pivoted. This modifi cation can be incorporated simply into a program; merely keep track of the number of pivots that take place during the course of the computation and then use Eq. (B9.1.3) to evalu- ate the determinant.

cha9792x_ch09_245-277.indd Page 263 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 263 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

264 GAUSS ELIMINATION

alerting the user. We will show the details of how this is done when we present a full algorithm for Gauss elimination later in this chapter.

9.4 TECHNIQUES FOR IMPROVING SOLUTIONS

The following techniques can be incorporated into the naive Gauss elimination algorithm to circumvent some of the pitfalls discussed in the previous section.

9.4.1 Use of More Signifi cant Figures

The simplest remedy for ill-conditioning is to use more signifi cant fi gures in the compu- tation. If your application can be extended to handle larger word size, such a feature will greatly reduce the problem. However, a price must be paid in the form of the computa- tional and memory overhead connected with using extended precision (recall Sec. 3.4.1).

9.4.2 Pivoting

As mentioned at the beginning of Sec. 9.3, obvious problems occur when a pivot element is zero because the normalization step leads to division by zero. Problems may also arise when the pivot element is close to, rather than exactly equal to, zero because if the magnitude of the pivot element is small compared to the other elements, then round-off errors can be introduced. Therefore, before each row is normalized, it is advantageous to determine the larg- est available coeffi cient in the column below the pivot element. The rows can then be switched so that the largest element is the pivot element. This is called partial pivoting. If columns as well as rows are searched for the largest element and then switched, the procedure is called complete pivoting. Complete pivoting is rarely used because switch- ing columns changes the order of the x’s and, consequently, adds signifi cant and usually unjustifi ed complexity to the computer program. The following example illustrates the advantages of partial pivoting. Aside from avoiding division by zero, pivoting also min- imizes round-off error. As such, it also serves as a partial remedy for ill-conditioning.

EXAMPLE 9.9 Partial Pivoting

Problem Statement. Use Gauss elimination to solve

0.0003x1 1 3.0000x2 5 2.0001

1.0000x1 1 1.0000x2 5 1.0000

Note that in this form the fi rst pivot element, a11 5 0.0003, is very close to zero. Then repeat the computation, but partial pivot by reversing the order of the equations. The exact solution is x1 5 1y3 and x2 5 2y3.

Solution. Multiplying the fi rst equation by 1y(0.0003) yields

x1 1 10,000x2 5 6667

which can be used to eliminate x1 from the second equation:

29999x2 5 26666

cha9792x_ch09_245-277.indd Page 264 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 264 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.4 TECHNIQUES FOR IMPROVING SOLUTIONS 265

which can be solved for

x2 5 2

3

This result can be substituted back into the fi rst equation to evaluate x1:

x1 5 2.0001 2 3(2y3)

0.0003 (E9.9.1)

However, due to subtractive cancellation, the result is very sensitive to the number of signifi cant fi gures carried in the computation:

Absolute Value of Percent Signifi cant Relative Error Figures x2 x1 for x1

3 0.667 23.33 1099 4 0.6667 0.0000 100 5 0.66667 0.30000 10 6 0.666667 0.330000 1 7 0.6666667 0.3330000 0.1

Note how the solution for x1 is highly dependent on the number of signifi cant fi gures. This is because in Eq. (E9.9.1), we are subtracting two almost-equal numbers. On the other hand, if the equations are solved in reverse order, the row with the larger pivot element is normalized. The equations are

1.0000x1 1 1.0000x2 5 1.0000

0.0003x1 1 3.0000x2 5 2.0001

Elimination and substitution yield x2 5 2y3. For different numbers of signifi cant fi gures, x1 can be computed from the fi rst equation, as in

x1 5 1 2 (2y3)

1 (E9.9.2)

This case is much less sensitive to the number of signifi cant fi gures in the computation:

Absolute Value of Percent Signifi cant Relative Error Figures x2 x1 for x1

3 0.667 0.333 0.1 4 0.6667 0.3333 0.01 5 0.66667 0.33333 0.001 6 0.666667 0.333333 0.0001 7 0.6666667 0.3333333 0.00001

Thus, a pivot strategy is much more satisfactory.

cha9792x_ch09_245-277.indd Page 265 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 265 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

266 GAUSS ELIMINATION

General-purpose computer programs must include a pivot strategy. Figure 9.5 provides a simple algorithm to implement such a strategy. Notice that the algorithm consists of two major loops. After storing the current pivot element and its row number as the variables, big and p, the first loop compares the pivot element with the elements below it to check whether any of these is larger than the pivot element. If so, the new largest element and its row number are stored in big and p. Then, the second loop switches the original pivot row with the one with the largest ele- ment so that the latter becomes the new pivot row. This pseudocode can be inte- grated into a program based on the other elements of Gauss elimination outlined in Fig. 9.4. The best way to do this is to employ a modular approach and write Fig. 9.5 as a subroutine (or procedure) that would be called directly after the beginning of the first loop in Fig. 9.4a. Note that the second IF/THEN construct in Fig. 9.5 physically interchanges the rows. For large matrices, this can become quite time consuming. Consequently, most codes do not actually exchange rows but rather keep track of the pivot rows by storing the ap- propriate subscripts in a vector. This vector then provides a basis for specifying the proper row ordering during the forward-elimination and back-substitution operations. Thus, the operations are said to be implemented in place.

9.4.3 Scaling

In Sec. 9.3.3, we proposed that scaling had value in standardizing the size of the deter- minant. Beyond this application, it has utility in minimizing round-off errors for those cases where some of the equations in a system have much larger coeffi cients than others. Such situations are frequently encountered in engineering practice when widely different units are used in the development of simultaneous equations. For instance, in electric- circuit problems, the unknown voltages can be expressed in units ranging from microvolts to kilovolts. Similar examples can arise in all fi elds of engineering. As long as each equation is consistent, the system will be technically correct and solvable. However, the use of widely differing units can lead to coeffi cients of widely differing magnitudes. This, in turn, can have an impact on round-off error as it affects pivoting, as illustrated by the following example.

EXAMPLE 9.10 Effect of Scaling on Pivoting and Round-Off

Problem Statement.

(a) Solve the following set of equations using Gauss elimination and a pivoting strategy:

2x1 1 100,000x2 5 100,000

x1 1    x2 5 2

(b) Repeat the solution after scaling the equations so that the maximum coeffi cient in each row is 1.

(c) Finally, use the scaled coeffi cients to determine whether pivoting is necessary. How- ever, actually solve the equations with the original coeffi cient values. For all cases, retain only three signifi cant fi gures. Note that the correct answers are x1 5 1.00002 and x2 5 0.99998 or, for three signifi cant fi gures, x1 5 x2 5 1.00.

p 5 k big 5 |ak,k| DOFOR ii 5 k11, n dummy 5 |aii,k| IF (dummy . big) big 5 dummy p 5 ii END IF END DO IF (p fi k) DOFOR jj 5 k, n dummy 5 ap,jj ap,jj 5 ak,jj ak,jj 5 dummy END DO dummy 5 bp bp 5 bk bk 5 dummy END IF

FIGURE 9.5 Pseudocode to implement partial pivoting.

cha9792x_ch09_245-277.indd Page 266 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 266 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.4 TECHNIQUES FOR IMPROVING SOLUTIONS 267

Solution.

(a) Without scaling, forward elimination is applied to give

2x1 1 100,000x2 5 100,000

250,000x2 5 250,000

which can be solved by back substitution for

x2 5 1.00

x1 5 0.00

Although x2 is correct, x1 is 100 percent in error because of round-off. (b) Scaling transforms the original equations to

0.00002x1 1 x2 5 1

x1 1 x2 5 2

Therefore, the rows should be pivoted to put the greatest value on the diagonal.

x1 1 x2 5 2

0.00002x1 1 x2 5 1

Forward elimination yields

x1 1 x2 5 2

x2 5 1.00

which can be solved for

x1 5 x2 5 1

Thus, scaling leads to the correct answer. (c) The scaled coeffi cients indicate that pivoting is necessary. We therefore pivot but

retain the original coeffi cients to give

x1 1 x2 5 2

2x1 1 100,000x2 5 100,000

Forward elimination yields

x1 1   x2 5 2

  100,000x2 5 100,000

which can be solved for the correct answer: x1 5 x2 5 1. Thus, scaling was useful in determining whether pivoting was necessary, but the equations themselves did not require scaling to arrive at a correct result.

cha9792x_ch09_245-277.indd Page 267 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 267 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

268 GAUSS ELIMINATION

FIGURE 9.6 Pseudocode to implement Gauss elimination with partial pivoting.

SUB Gauss (a, b, n, x, tol, er) DIMENSION s(n) er 5 0 DOFOR i 5 1, n si 5 ABS(ai,1) DOFOR j 5 2, n IF ABS(ai,j).si THEN si 5 ABS(ai,j) END DO END DO CALL Eliminate(a, s, n, b, tol, er) IF er ? 21 THEN CALL Substitute(a, n, b, x) END IF END Gauss

SUB Eliminate (a, s, n, b, tol, er) DOFOR k 5 1, n 2 1 CALL Pivot (a, b, s, n, k) IF ABS (ak,k/sk) , tol THEN er 5 21 EXIT DO END IF DOFOR i 5 k 1 1, n factor 5 ai,k/ak,k DOFOR j 5 k 1 1, n ai,j 5 ai,j 2 factor*ak,j END DO bi 5 bi 2 factor * bk END DO END DO IF ABS(an,n/sn) , to1 THEN er 5 21 END Eliminate

SUB Pivot (a, b, s, n, k) p 5 k big 5 ABS(ak,k/sk) DOFOR ii 5 k 1 1, n dummy 5 ABS(aii,k/sii) IF dummy . big THEN big 5 dummy p 5 ii END IF END DO IF p ? k THEN DOFOR jj 5 k, n dummy 5 ap,jj ap,jj 5 ak,jj ak,jj 5 dummy END DO dummy 5 bp bp 5 bk bk 5 dummy dummy 5 sp sp 5 sk sk 5 dummy END IF END pivot

SUB Substitute (a, n, b, x) xn 5 bn/an,n DOFOR i 5 n 2 1, 1, 21 sum 5 0 DOFOR j 5 i 1 1, n sum 5 sum 1 ai,j * xj END DO xn 5 (bn 2 sum) / an,n END DO END Substitute

cha9792x_ch09_245-277.indd Page 268 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 268 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.4 TECHNIQUES FOR IMPROVING SOLUTIONS 269

As in the previous example, scaling has utility in minimizing round-off. However, it should be noted that scaling itself also leads to round-off. For example, given the equation

2x1 1 300,000x2 5 1

and using three signifi cant fi gures, scaling leads to

0.00000667x1 1 x2 5 0.00000333

Thus, scaling introduces a round-off error to the fi rst coeffi cient and the right-hand-side constant. For this reason, it is sometimes suggested that scaling should be employed only as in part (c) of the preceding example. That is, it is used to calculate scaled values for the coeffi cients solely as a criterion for pivoting, but the original coeffi cient values are retained for the actual elimination and substitution computations. This involves a trade- off if the determinant is being calculated as part of the program. That is, the resulting determinant will be unscaled. However, because many applications of Gauss elimination do not require determinant evaluation, it is the most common approach and will be used in the algorithm in the next section.

9.4.4 Computer Algorithm for Gauss Elimination

The algorithms from Figs. 9.4 and 9.5 can now be combined into a larger algorithm to implement the entire Gauss elimination algorithm. Figure 9.6 shows an algorithm for a general subroutine to implement Gauss elimination. Note that the program includes modules for the three primary operations of the Gauss elimination algorithm: forward elimination, back substitution, and pivoting. In addition, there are several aspects of the code that differ and represent improvements over the pseudocodes from Figs. 9.4 and 9.5. These are:

The equations are not scaled, but scaled values of the elements are used to determine whether pivoting is to be implemented.

The diagonal term is monitored during the pivoting phase to detect near-zero occurrences in order to flag singular systems. If it passes back a value of er 5 21, a singular matrix has been detected and the computation should be terminated. A parameter tol is set by the user to a small number in order to detect near-zero occurrences.

EXAMPLE 9.11 Solution of Linear Algebraic Equations Using the Computer

Problem Statement. A computer program to solve linear algebraic equations such as one based on Fig. 9.6 can be used to solve a problem associated with the falling parachutist example discussed in Chap. 1. Suppose that a team of three parachutists is connected by a weightless cord while free-falling at a velocity of 5 m/s (Fig. 9.7).

cha9792x_ch09_245-277.indd Page 269 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 269 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

270 GAUSS ELIMINATION

Solution. Free-body diagrams for each of the parachutists are depicted in Fig. 9.8. Summing the forces in the vertical direction and using Newton’s second law gives a set of three simultaneous linear equations:

m1g 2 T 2 c1y     5 m1a

m2g 1 T 2 c2y 2 R 5 m2a

m3g   2 c3y 1 R 5 m3a

These equations have three unknowns: a, T, and R. After substituting the known values, the equations can be expressed in matrix form as (g 5 9.81 m/s2),

£ 70  1  0

60 21  1

40  0 21

§ • a

T

R

¶ 5 • 636.7

518.6

307.4

This system can be solved using your own software. The result is a 5 8.6041 m/s2; T 5 34.4118 N; and R 5 36.7647 N.

FIGURE 9.7 Three parachutists free-falling while connected by weightless cords.

R

T

1

2

3

a

T

m3g

R

TR m2g m1g

c3v c2v c1v

3 2 1

FIGURE 9.8 Free-body diagrams for each of the three falling parachutists.

Parachutist Mass, kg Drag Coeffi cient, kg/s

1 70 10 2 60 14 3 40 17

Calculate the tension in each section of cord and the acceleration of the team, given the following:

cha9792x_ch09_245-277.indd Page 270 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 270 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.6 NONLINEAR SYSTEMS OF EQUATIONS 271

9.5 COMPLEX SYSTEMS

In some problems, it is possible to obtain a complex system of equations

[C]{Z} 5 {W} (9.27)

where

[C] 5 [A] 1 i[B]

{Z} 5 {X} 1 i{Y}

{W} 5 {U} 1 i{V} (9.28)

where i 5 121. The most straightforward way to solve such a system is to employ one of the algo- rithms described in this part of the book, but replace all real operations with complex ones. Of course, this is only possible for those languages, such as Fortran, that allow complex variables. For languages that do not permit the declaration of complex variables, it is possible to write a code to convert real to complex operations. However, this is not a trivial task. An alternative is to convert the complex system into an equivalent one dealing with real variables. This can be done by substituting Eq. (9.28) into Eq. (9.27) and equating real and complex parts of the resulting equation to yield

[A]{X} 2 [B]{Y} 5 {U} (9.29)

and

[B]{X} 1 [A]{Y} 5 {V} (9.30)

Thus, the system of n complex equations is converted to a set of 2n real ones. This means that storage and execution time will be increased signifi cantly. Consequently, a trade-off exists regarding this option. If you evaluate complex systems infrequently, it is preferable to use Eqs. (9.29) and (9.30) because of their convenience. However, if you use them often and desire to employ a language that does not allow complex data types, it may be worth the up-front programming effort to write a customized equation solver that converts real to complex operations.

9.6 NONLINEAR SYSTEMS OF EQUATIONS

Recall that at the end of Chap. 6 we presented an approach to solve two nonlinear equa- tions with two unknowns. This approach can be extended to the general case of solving n simultaneous nonlinear equations.

f1(x1, x2, p , xn) 5 0 f2(x1, x2, p , xn) 5 0 . .

. . (9.31)

. .

fn(x1, x2, p , xn) 5 0

cha9792x_ch09_245-277.indd Page 271 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 271 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

272 GAUSS ELIMINATION

The solution of this system consists of the set of x values that simultaneously result in all the equations equaling zero. As described in Sec. 6.5.2, one approach to solving such systems is based on a multidimensional version of the Newton-Raphson method. Thus, a Taylor series expan- sion is written for each equation. For example, for the kth equation,

fk,i11 5 fk,i 1 (x1,i11 2 x1,i) 0fk,i 0x1

1 (x2,i11 2 x2,i) 0fk,i 0x2

1 p 1 (xn,i11 2 xn,i) 0fk,i 0xn

(9.32)

where the fi rst subscript, k, represents the equation or unknown and the second subscript denotes whether the value or function in question is at the present value (i) or at the next value (i 1 1). Equations of the form of (9.32) are written for each of the original nonlinear equa- tions. Then, as was done in deriving Eq. (6.20) from (6.19), all fk,i11 terms are set to zero as would be the case at the root, and Eq. (9.32) can be written as

2fk,i 1 x1,i 0fk,i 0x1

1 x2,i 0fk,i 0x2

1 p 1 xn,i 0fk,i 0xn

5 x1,i11 0fk,i 0x1

1 x2,i11 0fk,i 0x2

1 p 1 xn,i11 0fk,i 0xn

(9.33)

Notice that the only unknowns in Eq. (9.33) are the xk,i11 terms on the right-hand side. All other quantities are located at the present value (i) and, thus, are known at any iteration. Consequently, the set of equations generally represented by Eq. (9.33) (that is, with k 5 1, 2, . . . , n) constitutes a set of linear simultaneous equations that can be solved by methods elaborated in this part of the book. Matrix notation can be employed to express Eq. (9.33) concisely. The partial derivatives can be expressed as

[Z] 5 I

0f1,i 0x1

0f1,i 0x2

p 0f1,i 0xn

0f2,i 0x1

0f2,i 0x2

p 0f2,i 0xn

. . .

. . .

. . . 0fn,i 0x1

0fn,i 0x2

p 0fn,i 0xn

Y (9.34)

The initial and fi nal values can be expressed in vector form as

{Xi} T 5 :x1,i x2,i p  xn,i ;

and

{Xi11} T 5 :x1,i11 x2,i11 p xn,i11 ;

cha9792x_ch09_245-277.indd Page 272 15/10/13 9:08 PM f-479 cha9792x_ch09_245-277.indd Page 272 15/10/13 9:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

9.7 GAUSS-JORDAN 273

Finally, the function values at i can be expressed as

{Fi} T 5 : f1,i  f2,i p fn,i ;

Using these relationships, Eq. (9.33) can be represented concisely as

[Z]{Xi11} 5 2{Fi} 1 [Z]{Xi} (9.35)

Equation (9.35) can be solved using a technique such as Gauss elimination. This process can be repeated iteratively to obtain refi ned estimates in a fashion similar to the two- equation case in Sec. 6.5.2. It should be noted that there are two major shortcomings to the foregoing approach. First, Eq. (9.34) is often inconvenient to evaluate. Therefore, variations of the Newton- Raphson approach have been developed to circumvent this dilemma. As might be ex- pected, most are based on using fi nite-difference approximations for the partial derivatives that comprise [Z]. The second shortcoming of the multiequation Newton-Raphson method is that excel- lent initial guesses are usually required to ensure convergence. Because these are often diffi cult to obtain, alternative approaches that are slower than Newton-Raphson but which have better convergence behavior have been developed. One common approach is to reformulate the nonlinear system as a single function

F(x) 5 a n

i51 [

fi(x1, x2, p , xn)]2 (9.36)

where fi(xl, x2, . . . , xn) is the ith member of the original system of Eq. (9.31). The values of x that minimize this function also represent the solution of the nonlinear system. As we will see in Chap. 17, this reformulation belongs to a class of problems called nonlinear regression. As such, it can be approached with a number of optimization tech- niques such as the ones described later in this text (Part Four and specifi cally Chap. 14).

9.7 GAUSS-JORDAN

The Gauss-Jordan method is a variation of Gauss elimination. The major difference is that when an unknown is eliminated in the Gauss-Jordan method, it is eliminated from all other equations rather than just the subsequent ones. In addition, all rows are normalized by dividing them by their pivot elements. Thus, the elimination step results in an identity matrix rather than a triangular matrix (Fig. 9.9). Consequently, it is not necessary to em- ploy back substitution to obtain the solution. The method is best illustrated by an example.

EXAMPLE 9.12 Gauss-Jordan Method

Problem Statement. Use the Gauss-Jordan technique to solve the same system as in Example 9.5:

3x1 2 0.1x2 2 0.2x3 5 7.85

0.1x1 1 7x2 2 0.3x3 5 219.3

0.3x1 2 0.2x2 1 10x3 5 71.4

£ a11 a12 a13 b1 a21 a22 a23 b2 a31 a32 a33 b3

§

T

£ 1     0     0 b(n)1 0     1     0 b(n)2 0     0     1 b(n)3

§  

T x1 5 b(n)1

x2 5 b(n)2 x3 5 b(n)3

FIGURE 9.9 Graphical depiction of the Gauss-Jordan method. Compare with Fig. 9.3 to elucidate the differences between this tech- nique and Gauss elimination. The superscript (n) means that the elements of the right-hand- side vector have been modifi ed n times (for this case, n 5 3).

cha9792x_ch09_245-277.indd Page 273 15/10/13 9:09 PM f-479 cha9792x_ch09_245-277.indd Page 273 15/10/13 9:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

274 GAUSS ELIMINATION

Solution. First, express the coeffi cients and the right-hand side as an augmented matrix:

£ 3 20.1 20.2 7.85

0.1 7 20.3 219.3

0.3 20.2 10 71.4

§

Then normalize the fi rst row by dividing it by the pivot element, 3, to yield

£ 1 20.0333333 20.066667 2.61667

0.1 7 20.3 219.3

0.3 20.2 10 71.4

§

The x1 term can be eliminated from the second row by subtracting 0.1 times the fi rst row from the second row. Similarly, subtracting 0.3 times the fi rst row from the third row will eliminate the x1 term from the third row:

£ 1 20.0333333 20.066667 2.61667

0 7.00333 20.293333 219.5617

0 20.190000 10.0200 70.6150

§

Next, normalize the second row by dividing it by 7.00333:

£ 1 20.0333333 20.066667 2.61667

0 1 20.0418848 22.79320

0 20.190000 10.0200 70.6150

§

Reduction of the x2 terms from the fi rst and third equations gives

£ 1 0 20.0680629 2.52356

0 1 20.0418848 22.79320

0 0 10.01200 70.0843

§

The third row is then normalized by dividing it by 10.0120:

£ 1 0 20.0680629 2.52356

0 1 20.0418848 22.79320

0 0 1 7.0000

§

Finally, the x3 terms can be reduced from the fi rst and the second equations to give

£ 1 0 0 3.0000

0 1 0 22.5000

0 0 1 7.0000

§

Thus, as depicted in Fig. 9.9, the coeffi cient matrix has been transformed to the identity matrix, and the solution is obtained in the right-hand-side vector. Notice that no back substitution was required to obtain the solution.

All the material in this chapter regarding the pitfalls and improvements in Gauss elimination also applies to the Gauss-Jordan method. For example, a similar pivoting strategy can be used to avoid division by zero and to reduce round-off error.

cha9792x_ch09_245-277.indd Page 274 15/10/13 9:09 PM f-479 cha9792x_ch09_245-277.indd Page 274 15/10/13 9:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 275

Although the Gauss-Jordan technique and Gauss elimination might appear almost identical, the former requires more work. Using a similar approach to Sec. 9.2.1, it can be determined that the number of fl ops involved in naive Gauss-Jordan is

n3 1 n2 2 n ————Sas n increases n3 1 O(n2) (9.37)

Thus, Gauss-Jordan involves approximately 50 percent more operations than Gauss elim- ination [compare with Eq. (9.23)]. Therefore, Gauss elimination is the simple elimination method of preference for obtaining solutions of linear algebraic equations. One of the primary reasons that we have introduced the Gauss-Jordan, however, is that it is still used in engineering as well as in some numerical algorithms.

9.8 SUMMARY

In summary, we have devoted most of this chapter to Gauss elimination, the most fun- damental method for solving simultaneous linear algebraic equations. Although it is one of the earliest techniques developed for this purpose, it is nevertheless an extremely effective algorithm for obtaining solutions for many engineering problems. Aside from this practical utility, this chapter also provided a context for our discussion of general issues such as round-off, scaling, and conditioning. In addition, we briefl y presented material on the Gauss-Jordan method, as well as complex and nonlinear systems. Answers obtained using Gauss elimination may be checked by substituting them into the original equations. However, this does not always represent a reliable check for ill- conditioned systems. Therefore, some measure of condition, such as the determinant of the scaled system, should be computed if round-off error is suspected. Using partial pivoting and more signifi cant fi gures in the computation are two options for mitigating round-off error. In the next chapter, we will return to the topic of system condition when we discuss the matrix inverse.

PROBLEMS

9.1 (a) Write the following set of equations in matrix form:

8 5 6x3 1 2x2

2 2 x1 5 x3

5x2 1 8xl 5 13

(b) Multiply the matrix of coeffi cients by its transpose; i.e., [A][A]T. 9.2 A number of matrices are defi ned as

[A] 5 £ 4 7

1 2

5 6

§  [B] 5 £ 4 3 7

1 2 7

2 0 4

§

{C} 5 • 3

6

1

¶  [D] 5 c 9 4 3 26 2 21 7 5

d

[E] 5 £ 1 5 8

7 2 3

4 0 6

§

[F] 5 c 3 0 1 1 7 3

d   :G ; 5 :7 6 4 ; Answer the following questions regarding these matrices: (a) What are the dimensions of the matrices? (b) Identify the square, column, and row matrices. (c) What are the values of the elements: a12, b23, d32, e22, f12, g12? (d) Perform the following operations:

(1) [E] 1 [B] (5) [E] 3 [B]

(2) [A] 3 [F] (6) {C}T

(3) [B] 2 [E] (7) [B] 3 [A]

(4) 7 3 [B] (8) [D]T

cha9792x_ch09_245-277.indd Page 275 15/10/13 9:09 PM f-479 cha9792x_ch09_245-277.indd Page 275 15/10/13 9:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

276 GAUSS ELIMINATION

(e) Solve again, but with a11 modifi ed slightly to 0.52. Interpret your results.

9.8 Given the equations

10x1 1 2x2 2 x3 5 27

23x1 2 6x2 1 2x3 5 261.5

x1 1 x2 1 5x3 5 221.5

(a) Solve by naive Gauss elimination. Show all steps of the com- putation.

(b) Substitute your results into the original equations to check your answers.

9.9 Use Gauss elimination to solve:

8x1 1 2x2 2 2x3 5 22

10x1 1 2x2 1 4x3 5 4

12x1 1 2x2 1 2x3 5 6

Employ partial pivoting and check your answers by substituting them into the original equations. 9.10 Given the system of equations

23x2 1 7x3 5 2

x1 1 2x2 2 x3 5 3

5x1 2 2x2 5 2

(a) Compute the determinant. (b) Use Cramer’s rule to solve for the x’s. (c) Use Gauss elimination with partial pivoting to solve for the x’s. (d) Substitute your results back into the original equations to check

your solution. 9.11 Given the equations

2x1 2 6x2 2 x3 5 238

23x1 2 x2 1 7x3 5 234

28x1 1 x2 2 2x3 5 220

(a) Solve by Gauss elimination with partial pivoting. Show all steps of the computation.

(b) Substitute your results into the original equations to check your answers.

9.12 Use Gauss-Jordan elimination to solve:

2x1 1 x2 2 x3 5 1

5x1 1 2x2 1 2x3 5 24

3x1 1 x2 1 x3 5 5

Do not employ pivoting. Check your answers by substituting them into the original equations.

(9) [A] 3 {C} (11) [E]T[E]

(10) [I] 3 [B] (12) {C}T{C} 9.3 Three matrices are defi ned as

[A] 5 £ 1 6

3 10

7 4

§ [B] 5 c 1 3 0.5 2

d [C] 5 c 2 22 23 1

d

(a) Perform all possible multiplications that can be computed be- tween pairs of these matrices.

(b) Use the method in Box PT3.2 to justify why the remaining pairs cannot be multiplied.

(c) Use the results of (a) to illustrate why the order of multiplica- tion is important.

9.4 Use the graphical method to solve

4x1 2 8x2 5 224

2x1 1 6x2 5 34

Check your results by substituting them back into the equations. 9.5 Given the system of equations

21.1x1 1 10x2 5 120

22x1 1 17.4x2 5 174

(a) Solve graphically and check your results by substituting them back into the equations.

(b) On the basis of the graphical solution, what do you expect re- garding the condition of the system?

(c) Compute the determinant. (d) Solve by the elimination of unknowns. 9.6 For the set of equations

2x2 1 5x3 5 9

2x1 1 x2 1 x3 5 9

3x1 1 x2 5 10

(a) Compute the determinant. (b) Use Cramer’s rule to solve for the x’s. (c) Substitute your results back into the original equation to check

your results. 9.7 Given the equations

0.5x1 2 x2 5 29.5

1.02x1 2 2x2 5 218.8

(a) Solve graphically. (b) Compute the determinant. (c) On the basis of (a) and (b), what would you expect regarding

the system’s condition? (d) Solve by the elimination of unknowns.

cha9792x_ch09_245-277.indd Page 276 15/10/13 9:09 PM f-479 cha9792x_ch09_245-277.indd Page 276 15/10/13 9:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 277

9.17 Develop, debug, and test a program in either a high-level lan- guage or macro language of your choice to generate the transpose of a matrix. Test it on the matrices from Prob. 9.3. 9.18 Develop, debug, and test a program in either a high-level lan- guage or macro language of your choice to solve a system of equa- tions with Gauss elimination with partial pivoting. Base the program on the pseudocode from Fig. 9.6. Test the program using the following system (which has an answer of x1 5 x2 5 x3 5 1),

x1 1 2x2 2 x3 5 2

5x1 1 2x2 1 2x3 5 9

23x1 1 5x2 2 x3 5 1

9.19 Three masses are suspended vertically by a series of identi- cal springs where mass 1 is at the top and mass 3 is at the bottom. If g 5 9.81 m/s2, m1 5 2 kg, m2 5 3 kg, m3 5 2.5 kg, and the k’s 5 10 kg/s2, solve for the displacements x. 9.20 Develop, debug, and test a program in either a high-level lan- guage or macro language of your choice to solve a system of n si- multaneous nonlinear equations based on Sec. 9.6. Test the program by solving Prob. 7.12. 9.21 Recall from Sec. 8.2 that determining the chemistry of water exposed to atmospheric CO2 can be determined by solving fi ve nonlinear equations (Eqs. 8.6 through 8.10) for fi ve unknowns: cT, [HCO3

2], [CO3 22], [H1], and [OH2]. Employing the parameters

from Sec. 8.2 and the program developed in Prob. 9.20, solve this system for conditions in 1958 when the partial pressure of CO2 was 315 ppm. Use your results to compute the pH.

9.13 Solve:

x1 1 x2 2 x3 5 23

6x1 1 2x2 1 2x3 5 2

23x1 1 4x2 1 x3 5 1

with (a) naive Gauss elimination, (b) Gauss elimination with par- tial pivoting, and (c) Gauss-Jordan without partial pivoting. 9.14 Perform the same computation as in Example 9.11, but use fi ve parachutists with the following characteristics:

Parachutist Mass, kg Drag Coeffi cient, kg/s

1 55 10 2 75 12 3 60 15 4 75 16 5 90 10

The parachutists have a velocity of 9 m/s. 9.15 Solve

c 3 1 2i 4 2 i 1

d e z1 z2 f 5 e 2 1 i

3 f

9.16 Develop, debug, and test a program in either a high-level lan- guage or macro language of your choice to multiply two matrices— that is, [X] 5 [Y][Z], where [Y] is m by n and [Z] is n by p. Test the program using the matrices from Prob. 9.3.

cha9792x_ch09_245-277.indd Page 277 15/10/13 9:09 PM f-479 cha9792x_ch09_245-277.indd Page 277 15/10/13 9:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10 C H A P T E R 10

278

LU Decomposition and Matrix Inversion

This chapter deals with a class of elimination methods called LU decomposition tech- niques. The primary appeal of LU decomposition is that the time-consuming elimination step can be formulated so that it involves only operations on the matrix of coeffi cients, [A]. Thus, it is well suited for those situations where many right-hand-side vectors {B} must be evaluated for a single value of [A]. Although there are a variety of ways in which this is done, we will focus on showing how the Gauss elimination method can be imple- mented as an LU decomposition. One motive for introducing LU decomposition is that it provides an effi cient means to compute the matrix inverse. The inverse has a number of valuable applications in engineering practice. It also provides a means for evaluating system condition.

10.1 LU DECOMPOSITION

As described in Chap. 9, Gauss elimination is designed to solve systems of linear alge- braic equations,

[A]{X} 5 {B} (10.1)

Although it certainly represents a sound way to solve such systems, it becomes ineffi cient when solving equations with the same coeffi cients [A], but with different right-hand-side constants (the b’s). Recall that Gauss elimination involves two steps: forward elimination and back- substitution (Fig. 9.3). Of these, the forward-elimination step comprises the bulk of the computational effort (recall Table 9.1). This is particularly true for large systems of equations. LU decomposition methods separate the time-consuming elimination of the matrix [A] from the manipulations of the right-hand side {B}. Thus, once [A] has been “decom- posed,” multiple right-hand-side vectors can be evaluated in an effi cient manner. Interestingly, Gauss elimination itself can be expressed as an LU decomposition. Before showing how this can be done, let us fi rst provide a mathematical overview of the decomposition strategy.

cha9792x_ch10_278-299.indd Page 278 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 278 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.1 LU DECOMPOSITION 279

10.1.1 Overview of LU Decomposition

Just as was the case with Gauss elimination, LU decomposition requires pivoting to avoid division by zero. However, to simplify the following description, we will defer the issue of pivoting until after the fundamental approach is elaborated. In addition, the following explanation is limited to a set of three simultaneous equations. The results can be directly extended to n-dimensional systems. Equation (10.1) can be rearranged to give

[A]{X} 2 {B} 5 0 (10.2)

Suppose that Eq. (10.2) could be expressed as an upper triangular system:

£ u11 u12 u13 0 u22 u23 0 0 u33

§ • x1 x2 x3

¶ 5 • d1 d2 d3

¶ (10.3)

Recognize that this is similar to the manipulation that occurs in the fi rst step of Gauss elimination. That is, elimination is used to reduce the system to upper triangular form. Equation (10.3) can also be expressed in matrix notation and rearranged to give

[U]{X} 2 {D} 5 0 (10.4)

Now, assume that there is a lower diagonal matrix with 1’s on the diagonal,

[L] 5 £ 1 0 0

l21 1 0

l31 l32 1

§ (10.5)

that has the property that when Eq. (10.4) is premultiplied by it, Eq. (10.2) is the result. That is,

[L]{[U]{X} 2 {D}} 5 [A]{X} 2 {B} (10.6)

If this equation holds, it follows from the rules for matrix multiplication that

[L][U] 5 [A] (10.7)

and

[L]{D} 5 {B} (10.8)

A two-step strategy (see Fig. 10.1) for obtaining solutions can be based on Eqs. (10.4), (10.7), and (10.8):

1. LU decomposition step. [A] is factored or “decomposed” into lower [L] and upper [U] triangular matrices.

2. Substitution step. [L] and [U] are used to determine a solution {X} for a right-hand- side {B}. This step itself consists of two steps. First, Eq. (10.8) is used to generate an intermediate vector {D} by forward substitution. Then, the result is substituted into Eq. (10.4), which can be solved by back substitution for {X}.

Now, let us show how Gauss elimination can be implemented in this way.

cha9792x_ch10_278-299.indd Page 279 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 279 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

280 LU DECOMPOSITION AND MATRIX INVERSION

10.1.2 LU Decomposition Version of Gauss Elimination

Although it might appear at face value to be unrelated to LU decomposition, Gauss elimination can be used to decompose [A] into [L] and [U]. This can be easily seen for [U], which is a direct product of the forward elimination. Recall that the forward- elimination step is intended to reduce the original coeffi cient matrix [A] to the form

[U] 5 £ a11 a12 a13 0 a¿22 a¿23 0 0 a–33

§ (10.9)

which is in the desired upper triangular format. Though it might not be as apparent, the matrix [L] is also produced during the step. This can be readily illustrated for a three-equation system,

£ a11 a12 a13 a21 a22 a23 a31 a32 a33

§  • x1 x2 x3

¶ 5 • b1 b2 b3

The fi rst step in Gauss elimination is to multiply row 1 by the factor [recall Eq. (9.13)]

f 21 5 a21 a11

and subtract the result from the second row to eliminate a21. Similarly, row 1 is multiplied by

f 31 5 a31 a11

FIGURE 10.1 The steps in LU decomposition.

A X

X

X

B

B

D

D

DU

L

LU

Substitution

(b) Forward

(c) Backward

(a) Decomposition

cha9792x_ch10_278-299.indd Page 280 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 280 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.1 LU DECOMPOSITION 281

and the result subtracted from the third row to eliminate a31. The fi nal step is to multiply the modifi ed second row by

f 32 5 a¿32 a¿22

and subtract the result from the third row to eliminate a¿32. Now suppose that we merely perform all these manipulations on the matrix [A]. Clearly, if we do not want to change the equation, we also have to do the same to the right-hand side {B}. But there is absolutely no reason that we have to perform the ma- nipulations simultaneously. Thus, we could save the f’s and manipulate {B} later. Where do we store the factors f21, f31, and f32? Recall that the whole idea behind the elimination was to create zeros in a21, a31, and a32. Thus, we can store f21 in a21, f31 in a31, and f32 in a32. After elimination, the [A] matrix can therefore be written as

£ a11 a12 a13 f21 a¿22 a¿23 f31 f32 a–33

§ (10.10) This matrix, in fact, represents an effi cient storage of the LU decomposition of [A],

[A]S [L][U] (10.11)

where

[U] 5 £ a11 a12 a13 0 a¿22 a¿23 0 0 a–33

§ and

[L] 5 £ 1 0 0

f 21 1 0

f 31 f 32 1

§ The following example confi rms that [A] 5 [L][U].

EXAMPLE 10.1 LU Decomposition with Gauss Elimination

Problem Statement. Derive an LU decomposition based on the Gauss elimination per- formed in Example 9.5.

Solution. In Example 9.5, we solved the matrix

[A] 5 £ 3 20.1 20.2

0.1 7 20.3

0.3 20.2 10

§

After forward elimination, the following upper triangular matrix was obtained:

[U] 5 £ 3 20.1 20.2

0 7.00333 20.293333

0 0 10.0120

§

cha9792x_ch10_278-299.indd Page 281 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 281 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

282 LU DECOMPOSITION AND MATRIX INVERSION

The factors employed to obtain the upper triangular matrix can be assembled into a lower triangular matrix. The elements a21 and a31 were eliminated by using the factors

f 21 5 0.1

3 5 0.03333333  f 31 5

0.3

3 5 0.1000000

and the element a¿32 was eliminated by using the factor

f 32 5 20.19

7.00333 5 20.0271300

Thus, the lower triangular matrix is

[L] 5 £ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§ Consequently, the LU decomposition is

[A] 5 [L][U] 5 £ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§ £ 3 20.1 20.2

0 7.00333 2 0.293333

0 0 10.0120

§

This result can be verifi ed by performing the multiplication of [L][U] to give

[L][U] 5 £ 3 20.1 20.2

0.0999999 7 20.3

0.3 20.2 9.99996

§

where the minor discrepancies are due to round-off.

The following is pseudocode for a subroutine to implement the decomposition phase:

SUB Decompose (a, n) DOFOR k 5 1, n 2 1 DOFOR i 5 k 1 1, n factor 5 ai,k/ak,k ai,k 5 factor DOFOR j 5 k 1 1, n ai,j 5 ai,j 2 factor * ak,j END DO END DO END DO END Decompose

Notice that this algorithm is “naive” in the sense that pivoting is not included. This feature will be added later when we develop the full algorithm for LU decomposition. After the matrix is decomposed, a solution can be generated for a particular right- hand-side vector {B}. This is done in two steps. First, a forward-substitution step is executed by solving Eq. (10.8) for {D}. It is important to recognize that this merely

cha9792x_ch10_278-299.indd Page 282 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 282 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.1 LU DECOMPOSITION 283

amounts to performing the elimination manipulations on {B}. Thus, at the end of this step, the right-hand side will be in the same state that it would have been had we per- formed forward manipulation on [A] and {B} simultaneously. The forward-substitution step can be represented concisely as

di 5 bi 2 a i21

j51 ai jdj   for i 5 2, 3, p , n (10.12)

The second step then merely amounts to implementing back substitution, as in Eq. (10.4). Again, it is important to recognize that this is identical to the back-substitution phase of conventional Gauss elimination. Thus, in a fashion similar to Eqs. (9.16) and (9.17), the back-substitution step can be represented concisely as

xn 5 dnyann (10.13)

xi 5

di 2 a n

j5i11 ai j xj

aii    for i 5 n 2 1, n 2 2, p , 1 (10.14)

EXAMPLE 10.2 The Substitution Steps

Problem Statement. Complete the problem initiated in Example 10.1 by generating the fi nal solution with forward and back substitution.

Solution. As stated above, the intent of forward substitution is to impose the elimination manipulations, that we had formerly applied to [A], on the right-hand-side vector {B}. Recall that the system being solved in Example 9.5 was

£ 3 20.1 20.2

0.1 7 20.3

0.3 20.2 10

§ • x1 x2 x3

¶ 5 • 7.85

219.3

71.4

and that the forward-elimination phase of conventional Gauss elimination resulted in

£ 3 20.1 20.2

0 7.00333 20.293333

0 0 10.0120

§ • x1 x2 x3

¶ 5 • 7.85

219.5617

70.0843

¶ (E10.2.1)

The forward-substitution phase is implemented by applying Eq. (10.7) to our problem,

£ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§ • d1 d2 d3

¶ 5 • 7.85

219.3

71.4

or multiplying out the left-hand side,

d1           5 7.85

0.0333333d1 1     d2 5 219.3

  0.1d1 2 0.02713d2 1 d3 5 71.4

cha9792x_ch10_278-299.indd Page 283 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 283 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

284 LU DECOMPOSITION AND MATRIX INVERSION

We can solve the fi rst equation for d1,

d1 5 7.85

which can be substituted into the second equation to solve for

d2 5 219.3 2 0.0333333(7.85) 5 219.5617

Both d1 and d2 can be substituted into the third equation to give

d3 5 71.4 2 0.1(7.85) 1 0.02713(219.5617) 5 70.0843

Thus,

{D} 5 • 7.85

219.5617

70.0843

¶ which is identical to the right-hand side of Eq. (E10.2.1). This result can then be substituted into Eq. (10.4), [U]{X} 5 {D}, to give

£ 3 20.1 20.2

0 7.00333 20.293333

0 0 10.0120

§  • x1 x2 x3

¶ 5 • 7.85

219.5617

70.0843

¶ which can be solved by back substitution (see Example 9.5 for details) for the fi nal solution,

{X} 5 • 3

22.5

7.00003

The following is pseudocode for a subroutine to implement both substitution phases:

SUB Substitute (a, n, b, x) 'forward substitution DOFOR i 5 2, n sum 5 bi DOFOR j 5 1, i 2 1 sum 5 sum 2 ai,j * bj END DO bi 5 sum END DO 'back substitution xn 5 bn/an,n DOFOR i 5 n 2 1, 1, 21 sum 5 0 DOFOR j 5 i 1 1, n sum 5 sum 1 ai,j * xj END DO xi 5 (bi 2 sum)/ai,i END DO END Substitute

cha9792x_ch10_278-299.indd Page 284 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 284 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.1 LU DECOMPOSITION 285

The LU decomposition algorithm requires the same total multiply/divide fl ops as for Gauss elimination. The only difference is that a little less effort is expended in the de- composition phase since the operations are not applied to the right-hand side. Thus, the number of multiply/divide fl ops involved in the decomposition phase can be calculated as

n3

3 2

n

3 ————Sas n increases

n3

3 1 O(n) (10.15)

Conversely, the substitution phase takes a little more effort. Thus, the number of fl ops for forward and back substitution is n2. The total effort is therefore identical to Gauss elimination

n3

3 2

n

3 1 n2 ————Sas n increases

n3

3 1 O(n2) (10.16)

10.1.3 LU Decomposition Algorithm

An algorithm to implement an LU decomposition expression of Gauss elimination is listed in Fig. 10.2. Four features of this algorithm bear mention:

The factors generated during the elimination phase are stored in the lower part of the matrix. This can be done because these are converted to zeros anyway and are unnecessary for the final solution. This storage saves space.

This algorithm keeps track of pivoting by using an order vector o. This greatly speeds up the algorithm because only the order vector (as opposed to the whole row) is pivoted.

The equations are not scaled, but scaled values of the elements are used to determine whether pivoting is to be implemented.

The diagonal term is monitored during the pivoting phase to detect near-zero occurrences in order to flag singular systems. If it passes back a value of er 5 21, a singular matrix has been detected and the computation should be terminated. A parameter tol is set by the user to a small number in order to detect near-zero occurrences.

10.1.4 Crout Decomposition

Notice that for the LU decomposition implementation of Gauss elimination, the [L] matrix has 1’s on the diagonal. This is formally referred to as a Doolittle decomposition, or fac- torization. An alternative approach involves a [U] matrix with 1’s on the diagonal. This is called Crout decomposition. Although there are some differences between the approaches (Atkinson, 1978; Ralston and Rabinowitz, 1978), their performance is comparable. The Crout decomposition approach generates [U] and [L] by sweeping through the matrix by columns and rows, as depicted in Fig. 10.3. It can be implemented by the following concise series of formulas:

li, 1 5 ai, 1    for i 5 1, 2, p , n (10.17)

u1j 5 a1j

l11    for j 5 2, 3, p , n (10.18)

cha9792x_ch10_278-299.indd Page 285 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 285 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

286 LU DECOMPOSITION AND MATRIX INVERSION

For j 5 2, 3, . . . , n 2 1

li j 5 ai j 2 a j21

k51 likukj for i 5 j, j 1 1, p , n (10.19)

u j k 5

a j k 2 a j21

i51 ljiuik

lj j  for k 5 j 1 1, j 1 2, p , n (10.20)

SUB Ludecomp (a, b, n, tol, x, er) DIM on, sn er 5 0 CALL Decompose(a, n, tol, o, s, er) IF er ,. 21 THEN CALL Substitute(a, o, n, b, x) END IF END Ludecomp

SUB Decompose (a, n, tol, o, s, er) DOFOR i 5 1, n oi 5 i si 5 ABS(ai,1) DOFOR j 5 2, n IF ABS(ai,j).si THEN si 5 ABS(ai,j) END DO END DO DOFOR k 5 1, n 2 1 CALL Pivot(a, o, s, n, k) IF ABS(ao(k),kyso(k)) , tol THEN er 5 21 PRINT ao(k),kyso(k) EXIT DO END IF DOFOR i 5 k 1 1, n factor 5 ao(i),kyao(k),k ao(i),k 5 factor DOFOR j 5 k 1 1, n ao(i),j 5 ao(i),j 2 factor * ao(k),j END DO END DO END DO IF ABS(ao(k),kyso(k)) , tol THEN er 5 21 PRINT ao(k),kyso(k)

FIGURE 10.2 Pseudocode for an LU decomposition algorithm.

END IF END Decompose

SUB Pivot (a, o, s, n, k) p 5 k big 5 ABS(ao(k),kyso(k)) DOFOR ii 5 k 1 1, n dummy 5 ABS(ao(ii),kyso(ii)) IF dummy . big THEN big 5 dummy p 5 ii END IF END DO dummy 5 op op 5 ok ok 5 dummy END Pivot

SUB Substitute (a, o, n, b, x) DOFOR i 5 2, n sum 5 bo(i) DOFOR j 5 1, i 2 1 sum 5 sum 2 ao(i),j * bo(j) END DO bo(i) 5 sum END DO xn 5 bo(n)yao(n),n DOFOR i 5 n 2 1, 1, 21 sum 5 0 DOFOR j 5 i 1 1, n sum 5 sum 1 ao(i),j * xj END DO xi 5 (bo(i) 2 sum)yao(i),i END DO END Substitute

FIGURE 10.3 A schematic depicting the evaluations involved in Crout LU decomposition.

(a)

(b)

(c)

(d)

cha9792x_ch10_278-299.indd Page 286 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 286 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.2 THE MATRIX INVERSE 287

and

lnn 5 ann 2 a n21

k51 lnkukn (10.21)

Aside from the fact that it consists of a few concise loops, the foregoing approach also has the benefi t that storage space can be economized. There is no need to store the 1’s on the diagonal of [U] or the 0’s for [L] or [U] because they are givens in the method. Con- sequently, the values of [U] can be stored in the zero space of [L]. Further, close examina- tion of the foregoing derivation makes it clear that after each element of [A] is employed once, it is never used again. Therefore, as each element of [L] and [U] is computed, it can be substituted for the corresponding element (as designated by its subscripts) of [A]. Pseudocode to accomplish this is presented in Fig. 10.4. Notice that Eq. (10.17) is not included in the pseudocode because the fi rst column of [L] is already stored in [A]. Otherwise, the algorithm directly follows from Eqs. (10.18) through (10.21).

10.2 THE MATRIX INVERSE

In our discussion of matrix operations (Sec. PT3.2.2), we introduced the notion that if a matrix [A] is square, there is another matrix, [A]21, called the inverse of [A], for which [Eq. (PT3.3)]

[A][A21] 5 [A]21[A] 5 [I]

DOFOR j 5 2, n a1,j 5 a1,jya1,1 END DO DOFOR j 5 2, n 2 1 DOFOR i 5 j, n sum 5 0 DOFOR k 5 1, j 2 1 sum 5 sum 1 ai,k ? ak,j END DO ai,j 5 ai,j 2 sum END DO DOFOR k 5 j 1 1, n sum 5 0 DOFOR i 5 1, j 2 1 sum 5 sum 1 aj,i ? ai,k END DO aj,k 5 (aj,k 2 sum)yaj,j END DO END DO sum 5 0 DOFOR k 5 1, n 2 1 sum 5 sum 1 an,k ? ak,n END DO an,n 5 an,n 2 sum

FIGURE 10.4 Pseudocode for Crout’s LU decomposition algorithm.

cha9792x_ch10_278-299.indd Page 287 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 287 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

288 LU DECOMPOSITION AND MATRIX INVERSION

Now we will focus on how the inverse can be computed numerically. Then we will explore how it can be used for engineering analysis.

10.2.1 Calculating the Inverse

The inverse can be computed in a column-by-column fashion by generating solutions with unit vectors as the right-hand-side constants. For example, if the right-hand-side constant has a 1 in the fi rst position and zeros elsewhere,

{b} 5 • 1

0

0

the resulting solution will be the fi rst column of the matrix inverse. Similarly, if a unit vector with a 1 at the second row is used

{b} 5 • 0

1

0

the result will be the second column of the matrix inverse. The best way to implement such a calculation is with the LU decomposition algorithm described at the beginning of this chapter. Recall that one of the great strengths of LU decomposition is that it provides a very effi cient means to evaluate multiple right- hand-side vectors. Thus, it is ideal for evaluating the multiple unit vectors needed to compute the inverse.

EXAMPLE 10.3 Matrix Inversion

Problem Statement. Employ LU decomposition to determine the matrix inverse for the system from Example 10.2.

[A] 5 £ 3 20.1 20.2

0.1 7 20.3

0.3 20.2 10

§

Recall that the decomposition resulted in the following lower and upper triangular matrices:

[U] 5 £ 3 20.1 20.2

0 7.00333 20.293333

0 0 10.0120

§ [L] 5 £ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§

Solution. The fi rst column of the matrix inverse can be determined by performing the forward-substitution solution procedure with a unit vector (with 1 in the fi rst row) as the right-hand-side vector. Thus, Eq. (10.8), the lower-triangular system, can be set up as

£ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§ • d1 d2 d3

¶ 5 • 1

0

0

cha9792x_ch10_278-299.indd Page 288 24/10/13 3:48 PM F-468 cha9792x_ch10_278-299.indd Page 288 24/10/13 3:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.2 THE MATRIX INVERSE 289

and solved with forward substitution for {D}T 5 :1 20.03333 20.1009 ; . This vector can then be used as the right-hand side of Eq. (10.3),

£ 3 20.1 20.2

0 7.00333 20.293333

0 0 10.0120

§ • x1 x2 x3

¶ 5 • 1

20.03333

20.1009

which can be solved by back substitution for {X}T 5 :0.33249 20.00518 20.01008 ; , which is the fi rst column of the matrix,

[A]21 5 £ 0.33249 0 0

20.00518 0 0

20.01008 0 0

§

To determine the second column, Eq. (10.8) is formulated as

£ 1 0 0

0.0333333 1 0

0.100000 20.0271300 1

§ • d1 d2 d3

¶ 5 • 0

1

0

This can be solved for {D}, and the results are used with Eq. (10.3) to determine {X}T 5 :0.004944 0.142903 0.00271 ; , which is the second column of the matrix,

[A]21 5 £ 0.33249 0.004944 0

20.00518 0.142903 0

20.01008 0.00271 0

§

Finally, the forward- and back-substitution procedures can be implemented with {B}T 5 :0 0 1 ; to solve for {X}T 5 :0.006798 0.004183 0.09988 ; , which is the fi nal column of the matrix,

[A]21 5 £ 0.33249 0.004944 0.006798

20.00518 0.142903 0.004183

20.01008 0.00271 0.09988

§

The validity of this result can be checked by verifying that [A][A]21 5 [I].

Pseudocode to generate the matrix inverse is shown in Fig. 10.5. Notice how the decomposition subroutine from Fig. 10.2 is called to perform the decomposition and then generates the inverse by repeatedly calling the substitution algorithm with unit vectors. The effort required for this algorithm is simply computed as

n3

3 2

n

3    1  n(n2)   5

4n3

3 2

n

4 (10.22)

decomposition 1 n 3 substitutions

where from Sec. 10.1.2, the decomposition is defi ned by Eq. (10.15) and the effort in- volved with every right-hand-side evaluation involves n2 multiply/divide fl ops.

cha9792x_ch10_278-299.indd Page 289 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 289 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

290 LU DECOMPOSITION AND MATRIX INVERSION

10.2.2 Stimulus-Response Computations

As discussed in Sec. PT3.1.2, many of the linear systems of equations confronted in engi- neering practice are derived from conservation laws. The mathematical expression of these laws is some form of balance equation to ensure that a particular property—mass, force, heat, momentum, or other—is conserved. For a force balance on a structure, the properties might be horizontal or vertical components of the forces acting on each node of the structure (see Sec. 12.2). For a mass balance, the properties might be the mass in each reactor of a chemical process (see Sec. 12.1). Other fi elds of engineering would yield similar examples. A single balance equation can be written for each part of the system, resulting in a set of equations defi ning the behavior of the property for the entire system. These equa- tions are interrelated, or coupled, in that each equation may include one or more of the variables from the other equations. For many cases, these systems are linear and, there- fore, of the exact form dealt with in this chapter:

[A]{X} 5 {B} (10.23)

Now, for balance equations, the terms of Eq. (10.23) have a defi nite physical interpreta- tion. For example, the elements of {X} are the levels of the property being balanced for each part of the system. In a force balance of a structure, they represent the horizontal and vertical forces in each member. For the mass balance, they are the mass of chemical in each reactor. In either case, they represent the system’s state or response, which we are trying to determine. The right-hand-side vector {B} contains those elements of the balance that are in- dependent of behavior of the system—that is, they are constants. As such, they often represent the external forces or stimuli that drive the system.

FIGURE 10.5 Driver program that uses some of the subprograms from Fig. 10.2 to generate a matrix inverse.

CALL Decompose (a, n, tol, o, s, er) IF er 5 0 THEN DOFOR i 5 1, n DOFOR j 5 1, n IF i 5 j THEN b(j) 5 1 ELSE b(j) 5 0 END IF END DO CALL Substitute (a, o, n, b, x) DOFOR j 5 1, n ai(j, i) 5 x(j) END DO END DO Output ai, if desired ELSE PRINT "ill-conditioned system" END IF

cha9792x_ch10_278-299.indd Page 290 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 290 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.3 ERROR ANALYSIS AND SYSTEM CONDITION 291

Finally, the matrix of coeffi cients [A] usually contains the parameters that express how the parts of the system interact or are coupled. Consequently, Eq. (10.23) might be reexpressed as

[Interactions]{response} 5 {stimuli}

Thus, Eq. (10.23) can be seen as an expression of the fundamental mathematical model that we formulated previously as a single equation in Chap. 1 [recall Eq. (1.1)]. We can now see that Eq. (10.23) represents a version that is designed for coupled systems involv- ing several dependent variables {X}. As we know from this chapter and Chap. 9, there are a variety of ways to solve Eq. (10.23). However, using the matrix inverse yields a particularly interesting result. The formal solution can be expressed as

{X} 5 [A]21{B}

or (recalling our defi nition of matrix multiplication from Box PT3.2)

x1 5 a 21 11 b1 1 a

21 12 b2 1 a

21 13 b3

x2 5 a 21 21 b1 1 a

21 22 b2 1 a

21 23 b3

x3 5 a 21 31 b1 1 a

21 32 b2 1 a

21 33 b3

Thus, we fi nd that the inverted matrix itself, aside from providing a solution, has ex- tremely useful properties. That is, each of its elements represents the response of a single part of the system to a unit stimulus of any other part of the system. Notice that these formulations are linear and, therefore, superposition and propor- tionality hold. Superposition means that if a system is subject to several different stimuli (the b’s), the responses can be computed individually and the results summed to obtain a total response. Proportionality means that multiplying the stimuli by a quantity results in the response to those stimuli being multiplied by the same quantity. Thus, the coef- fi cient a2111 is a proportionality constant that gives the value of x1 due to a unit level of b1. This result is independent of the effects of b2 and b3 on x1, which are refl ected in the coeffi cients a2112 and a

21 13 , respectively. Therefore, we can draw the general conclusion

that the element a21ij of the inverted matrix represents the value of xi due to a unit quan- tity of bj. Using the example of the structure, element a

21 ij of the matrix inverse would

represent the force in member i due to a unit external force at node j. Even for small systems, such behavior of individual stimulus-response interactions would not be intui- tively obvious. As such, the matrix inverse provides a powerful technique for understand- ing the interrelationships of component parts of complicated systems. This power will be demonstrated in Secs. 12.1 and 12.2.

10.3 ERROR ANALYSIS AND SYSTEM CONDITION

Aside from its engineering applications, the inverse also provides a means to discern whether systems are ill-conditioned. Three methods are available for this purpose:

1. Scale the matrix of coeffi cients [A] so that the largest element in each row is 1. Invert the scaled matrix and if there are elements of [A]21 that are several orders of magnitude greater than one, it is likely that the system is ill-conditioned (see Box 10.1).

cha9792x_ch10_278-299.indd Page 291 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 291 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

292 LU DECOMPOSITION AND MATRIX INVERSION

2. Multiply the inverse by the original coeffi cient matrix and assess whether the result is close to the identity matrix. If not, it indicates ill-conditioning.

3. Invert the inverted matrix and assess whether the result is suffi ciently close to the original coeffi cient matrix. If not, it again indicates that the system is ill-conditioned.

Although these methods can indicate ill-conditioning, it would be preferable to ob- tain a single number (such as the condition number from Sec. 4.2.3) that could serve as an indicator of the problem. Attempts to formulate such a matrix condition number are based on the mathematical concept of the norm.

10.3.1 Vector and Matrix Norms

A norm is a real-valued function that provides a measure of the size or “length” of multicomponent mathematical entities such as vectors and matrices (see Box 10.2). A simple example is a vector in three-dimensional Euclidean space (Fig. 10.6) that can be represented as

:F ; 5 :a b c ; where a, b, and c are the distances along the x, y, and z axes, respectively. The length of this vector—that is, the distance from the coordinate (0, 0, 0) to (a, b, c)—can be simply computed as

BFBe 5 2a 2 1 b2 1 c2

where the nomenclature BFBe indicates that this length is referred to as the Euclidean norm of [F].

Box 10.1 Interpreting the Elements of the Matrix Inverse as a Measure of Ill-Conditioning

One method for assessing a system’s condition is to scale [A] so that the largest element in each row is 1 and then compute [A]21. If elements of [A]21 are several orders of magnitude greater than the elements of the original scaled matrix, it is likely that the system is ill-conditioned. Insight into this approach can be gained by recalling that a way to check whether an approximate solution {X} is acceptable is to substitute it into the original equations and see whether the origi- nal right-hand-side constants result. This is equivalent to

{R} 5 {B} 2 [A]{X̃} (B10.1.1)

where {R} is the residual between the right-hand-side constants and the values computed with the solution {X̃}. If {R} is small, we might conclude that the {X̃} values are adequate. However, suppose that {X} is the exact solution that yields a zero residual, as in

{0} 5 {B} 2 [A]{X} (B10.1.2)

Subtracting Eq. (B10.1.2) from (B10.1.1) yields

{R} 5 [A] {X} 2 {X̃}

Multiplying both sides of this equation by [A]21 gives

{X} 2 {X̃} 5 [A]21{R}

This result indicates why checking a solution by substitution can be misleading. For cases where elements of [A]21 are large, a small discrepancy in the right-hand-side residual {R} could cor- respond to a large error {X} 2 {X̃} in the calculated value of the unknowns. In other words, a small residual does not guarantee an accurate solution. However, we can conclude that if the largest element of [A]21 is on the order of magnitude of unity, the system can be considered to be well-conditioned. Conversely, if [A]21 includes elements much larger than unity, we conclude that the system is ill-conditioned.

cha9792x_ch10_278-299.indd Page 292 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 292 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.3 ERROR ANALYSIS AND SYSTEM CONDITION 293

Box 10.2 Matrix Norms

As developed in this section, Euclidean norms can be employed to quantify the size of a vector,

B X Be 5 B a n

i51 x2i

or matrix,

B A Be 5 B a n

i51 a

n

j51 a2i, j

For vectors, there are alternatives called p norms that can be represented generally by

B X Bp 5 aa n

i51 Zxi Z

pb1yp

We can also see that the Euclidean norm and the 2 norm, B XB2, are identical for vectors. Other important examples are

B X B1 5 a n

i51 Zxi Z

which represents the norm as the sum of the absolute values of the elements. Another is the maximum-magnitude or uniform-vector norm.

B X Bq 5 max 1#i#n

Zxi Z

which defi nes the norm as the element with the largest absolute value. Using a similar approach, norms can be developed for matrices. For example,

B A B1 5 max 1#j#n a

n

i51 Zaij Z

That is, a summation of the absolute values of the coeffi cients is performed for each column, and the largest of these summations is taken as the norm. This is called the column-sum norm. A similar determination can be made for the rows, resulting in a uniform-matrix or row-sum norm,

B A Bq 5 max 1#i#n a

n

j51 Zaij Z

It should be noted that, in contrast to vectors, the 2 norm and the Euclidean norm for a matrix are not the same. Whereas the Euclidean norm B ABe can be easily determined by Eq. (10.24), the matrix 2 norm B AB2 is calculated as

B AB2 5 (mmax) 1y2

where mmax is the largest eigenvalue of [A] T [A]. In Chap. 27, we

will learn more about eigenvalues. For the time being, the impor- tant point is that the B AB2, or spectral, norm is the minimum norm and, therefore, provides the tightest measure of size (Ortega 1972).

FIGURE 10.6 Graphical depiction of a vector :F ; 5 :a b c ; in Euclidean space.

y

x

a 2 �

b 2 �

c 2

b

�F � e=

z

c

a

cha9792x_ch10_278-299.indd Page 293 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 293 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

294 LU DECOMPOSITION AND MATRIX INVERSION

Similarly, for an n-dimensional vector :X ; 5 :x1 x2 p xn ; , a Euclidean norm would be computed as

B X Be 5 B a n

i51 x2i

The concept can be extended further to a matrix [A], as in

B A Be 5 B a n

i51 a

n

j51 a2i, j (10.24)

which is given a special name—the Frobenius norm. However, as with the other vector norms, it provides a single value to quantify the “size” of [A]. It should be noted that there are alternatives to the Euclidean and Frobenius norms (see Box 10.2). For example, a uniform vector norm is defi ned as

B X Bq 5 max 1#i#n

Zxi Z

That is, the element with the largest absolute value is taken as the measure of the vector’s size. Similarly, a uniform matrix norm or row-sum norm is defi ned as

B ABq 5 max 1#i#n a

n

j51 Zaij Z (10.25)

In this case, the sum of the absolute value of the elements is computed for each row, and the largest of these is taken as the norm. Although there are theoretical benefi ts for using certain of the norms, the choice is sometimes infl uenced by practical considerations. For example, the uniform-row norm is widely used because of the ease with which it can be calculated and the fact that it usu- ally provides an adequate measure of matrix size.

10.3.2 Matrix Condition Number

Now that we have introduced the concept of the norm, we can use it to defi ne

Cond [A] 5 B AB # B A21B (10.26)

where Cond [A] is called the matrix condition number. Note that for a matrix [A], this number will be greater than or equal to 1. It can be shown (Ralston and Rabinowitz, 1978; Gerald and Wheatley, 2004) that

B¢X B B X B

# Cond [A] B¢AB BAB

That is, the relative error of the norm of the computed solution can be as large as the relative error of the norm of the coeffi cients of [A] multiplied by the condition number. For example, if the coeffi cients of [A] are known to t-digit precision (that is, rounding errors are on the order of 102t) and Cond [A] 5 10c, the solution [X] may be valid to only t 2 c digits (rounding errors ,10c2t).

cha9792x_ch10_278-299.indd Page 294 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 294 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

10.3 ERROR ANALYSIS AND SYSTEM CONDITION 295

EXAMPLE 10.4 Matrix Condition Evaluation

Problem Statement. The Hilbert matrix, which is notoriously ill-conditioned, can be represented generally as

F 1 1y2 1y3 p 1yn1y2 1y3 1y4 p 1y(n 1 1). . . . . . . .

. . . .

1yn 1y(n 1 1) 1y(n 1 2) p 1y(2n 2 1)

V Use the row-sum norm to estimate the matrix condition number for the 3 3 3 Hilbert matrix,

[A] 5 £ 1 1y2 1y3

1y2 1y3 1y4 1y3 1y4 1y5

§

Solution. First, the matrix can be normalized so that the maximum element in each row is 1,

[A] 5 £ 1 1y2 1y3 1 2y3 1y2 1 3y4 3y5

§

Summing each of the rows gives 1.833, 2.1667, and 2.35. Thus, the third row has the largest sum and the row-sum norm is

B ABq 5 1 1 3

4 1

3

5 5 2.35

The inverse of the scaled matrix can be computed as

[A]21 5 £ 9 218 10

236 96 260

30 290 60

§

Note that the elements of this matrix are larger than the original matrix. This is also refl ected in its row-sum norm, which is computed as

BA21Bq 5 Z236 Z 1 Z96 Z 1 Z260 Z 5 192

Thus, the condition number can be calculated as

Cond [A] 5 2.35(192) 5 451.2

The fact that the condition number is considerably greater than unity suggests that the system is ill-conditioned. The extent of the ill-conditioning can be quantifi ed by calculating c 5 log 451.2 5 2.65. Computers using IEEE fl oating-point representation

cha9792x_ch10_278-299.indd Page 295 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 295 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

296 LU DECOMPOSITION AND MATRIX INVERSION

have approximately t 5 log 2224 5 7.2 signifi cant base-10 digits (recall Sec. 3.4.1). Therefore, the solution could exhibit rounding errors of up to 10(2.65-7.2) 5 3 3 1025. Note that such estimates almost always overpredict the actual error. However, they are useful in alerting you to the possibility that round-off errors may be signifi cant.

Practically speaking, the problem with implementing Eq. (10.26) is the computa- tional price required to obtain B A21B. Rice (1983) outlines some possible strategies to mitigate this problem. Further, he suggests an alternative way to assess system condi- tion: run the same solution on two different compilers. Because the resulting codes will likely implement the arithmetic differently, the effect of ill-conditioning should be evi- dent from such an experiment. Finally, it should be mentioned that software packages such as MATLAB software and Mathcad have the capability to conveniently compute matrix condition. We will review these capabilities when we review such packages at the end of Chap. 11.

10.3.3 Iterative Refi nement

In some cases, round-off errors can be reduced by the following procedure. Suppose that we are solving the following set of equations:

a11x1 1 a12x2 1 a13x3 5 b1

a21x1 1 a22x2 1 a23x3 5 b2 (10.27)

a31x1 1 a32x2 1 a33x3 5 b3

For conciseness, we will limit the following discussion to this small (3 3 3) system. However, the approach is generally applicable to larger sets of linear equations. Suppose an approximate solution vector is given by {X̃}T 5 : x̃1 x̃2 x̃3 ; . This solution can be substituted into Eq. (10.27) to give

a11x̃1 1 a12x̃2 1 a13x̃3 5 b̃1

a21x̃1 1 a22x̃2 1 a23x̃3 5 b̃2 (10.28)

a31x̃1 1 a32x̃2 1 a33x̃3 5 b̃3

Now, suppose that the exact solution {X} is expressed as a function of the approximate solution and a vector of correction factors {DX}, where

x1 5 x̃1 1 ¢x1 x2 5 x̃2 1 ¢x2 (10.29) x3 5 x̃3 1 ¢x3

If these results are substituted into Eq. (10.27), the following system results:

a11(x̃1 1 ¢x1) 1 a12(x̃2 1 ¢x2) 1 a13(x̃3 1 ¢x3) 5 b1 a21(x̃1 1 ¢x1) 1 a22(x̃2 1 ¢x2) 1 a23(x̃3 1 ¢x3) 5 b2 (10.30) a31(x̃1 1 ¢x1) 1 a32(x̃2 1 ¢x2) 1 a33(x̃3 1 ¢x3) 5 b3

cha9792x_ch10_278-299.indd Page 296 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 296 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 297

Now, Eq. (10.28) can be subtracted from Eq. (10.30) to yield

a11¢x1 1 a12¢x2 1 a13¢x3 5 b1 2 b̃1 5 E1 a21¢x1 1 a22¢x2 1 a23¢x3 5 b2 2 b̃2 5 E2 (10.31) a31¢x1 1 a32¢x2 1 a33¢x3 5 b3 2 b̃3 5 E3

This system itself is a set of simultaneous linear equations that can be solved to obtain the correction factors. The factors can then be applied to improve the solution, as specifi ed by Eq. (10.29). It is relatively straightforward to integrate an iterative refi nement procedure into com- puter programs for elimination methods. It is especially effective for the LU decomposition approaches described earlier, which are designed to evaluate different right-hand-side vec- tors effi ciently. Note that to be effective for correcting ill-conditioned systems, the E’s in Eq. (10.31) must be expressed in double precision.

PROBLEMS

10.1 Use the rules of matrix multiplication to prove that Eqs. (10.7) and (10.8) follow from Eq. (10.6). 10.2 (a) Use naive Gauss elimination to decompose the following system according to the description in Sec. 10.1.2.

10x1 1 2x2 2 x3 5 27

23x1 2 6x2 1 2x3 5 261.5

x1 1 x2 1 5x3 5 221.5

Then, multiply the resulting [L] and [U] matrices to determine that [A] is produced. (b) Use LU decomposition to solve the system. Show all the steps in the computation. (c) Also solve the system for an alternative right-hand-side vector: {B}T 5 :12 18 26 ; . 10.3 (a) Solve the following system of equations by LU decomposition

without pivoting

8x1 1 4x2 2 x3 5 11

22x1 1 5x2 1 x3 5 4

2x1 2 x2 1 6x3 5 7

(b) Determine the matrix inverse. Check your results by verifying that [A][A]21 5 [I].

10.4 Solve the following system of equations using LU decompo- sition with partial pivoting:

2x1 2 6x2 2 x3 5 238

23x1 2 x2 1 7x3 5 234

28x1 1 x2 2 2x3 5 220

10.5 Determine the total fl ops as a function of the number of equations n for the (a) decomposition, (b) forward-substitution, and (c) back-substitution phases of the LU decomposition version of Gauss elimination.

10.6 Use LU decomposition to determine the matrix inverse for the following system. Do not use a pivoting strategy, and check your results by verifying that [A][A]21 5 [I].

10x1 1 2x2 2 x3 5 27

23x1 2 6x2 1 2x3 5 261.5

x1 1 x2 1 5x3 5 221.5

10.7 Perform Crout decomposition on

2x1 2 5x2 1 x3 5 12

2x1 1 3x2 2 x3 5 28

3x1 2 4x2 1 2x3 5 16

Then, multiply the resulting [L] and [U] matrices to determine that [A] is produced. 10.8 The following system of equations is designed to determine concentrations (the c’s in gym3) in a series of coupled reactors as a function of the amount of mass input to each reactor (the right-hand sides in gyday),

15c1 2 3c2 2 c3 5 3800

23c1 1 18c2 2 6c3 5 1200

24c1 2 c2 1 12c3 5 2350

(a) Determine the matrix inverse. (b) Use the inverse to determine the solution. (c) Determine how much the rate of mass input to reactor 3 must be

increased to induce a 10 g/m3 rise in the concentration of reactor 1. (d) How much will the concentration in reactor 3 be reduced if the

rate of mass input to reactors 1 and 2 is reduced by 500 and 250 g/day, respectively?

cha9792x_ch10_278-299.indd Page 297 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 297 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

298 LU DECOMPOSITION AND MATRIX INVERSION

How many digits of precision will be lost due to ill-conditioning? (b) Repeat (a), but scale the matrix by making the maximum ele- ment in each row equal to one. 10.16 Determine the condition number based on the row-sum norm for the normalized 5 3 5 Hilbert matrix. How many signifi - cant digits of precision will be lost due to ill-conditioning? 10.17 Besides the Hilbert matrix, there are other matrices that are inherently ill-conditioned. One such case is the Vandermonde matrix, which has the following form:

£ x21 x1 1

x2 2 x2 1

x23 x3 1

§

(a) Determine the condition number based on the row-sum norm for the case where x1 5 4, x2 5 2, and x3 5 7.

(b) Use MATLAB or Mathcad software to compute the spectral and Frobenius condition numbers.

10.18 Develop a user-friendly program for LU decomposition based on the pseudocode from Fig. 10.2. 10.19 Develop a user-friendly program for LU decomposition, in- cluding the capability to evaluate the matrix inverse. Base the pro- gram on Figs. 10.2 and 10.5. 10.20 Use iterative refi nement techniques to improve x1 5 2, x2 5 23, and x3 5 8, which are approximate solutions of

2x1 1 5x2 1 x3 5 25

5x1 1 2x2 1 x3 5 12

x1 1 2x2 1 x3 5 3

10.21 Consider vectors:

A S

5 2i S

2 3 j S

1 ak S

B S

5 bi S

1 j S

2 4k S

C S

5 3i S

1 c j S

1 2k S

Vector A S

is perpendicular to B S

as well as to C S

. It is also known that B

S # C S

5 2. Use any method studied in this chapter to solve for the three unknowns, a, b, and c. 10.22 Consider the following vectors:

A S

5 a i S

1 b j S

1 ck S

B S

5 22 i S

1 j S

2 4k S

C S

5 i S

1 3 j S

1 2k S

where A S

is an unknown vector. If

(A S

3 B S

) 1 (A S

3 C S

) 5 (5a 1 6) i S

1 (3b 2 2) j S

1 (24c 1 1) k S

use any method learned in this chapter to solve for the three un- knowns, a, b, and c.

10.9 Solve the following set of equations with LU decomposition:

3x1 2 2x2 1 x3 5 210

2x1 1 6x2 2 4x3 5 44

2x1 2 2x2 1 5x3 5 226

10.10 (a) Determine the LU decomposition without pivoting by hand for the following matrix and check your results by validating that [L][U] 5 [A].

£ 8 2 1

3 7 2

2 3 9

§

(b) Employ the result of (a) to compute the determinant. (c) Repeat (a) and (b) using MATLAB. 10.11 Use the following LU decomposition to (a) compute the de- terminant and (b) solve [A]{x} 5 {b} with {b}T 5 :210 44 226=.

[A] 5 [L] [U] 5 £  1

0.6667   1

20.3333 20.3636 1

§ £ 3  22   1

 7.3333 24.6667

     3.6364

§

10.12 Determine B A Be, B A B1, and B ABq for

[A] 5 £ 8 2 210

29 1 3

15 21 6

§

Scale the matrix by making the maximum element in each row equal to one. 10.13 Determine the Frobenius and the row-sum norms for the systems in Probs. 10.3 and 10.4. Scale the matrices by making the maximum element in each row equal to one. 10.14 A matrix [A] is defi ned as

[A] 5 ≥ 0.125 0.25 0.5 1

0.015625 0.625 0.25 1

0.00463 0.02777 0.16667 1

0.001953 0.015625 0.125 1

¥

Using the column-sum norm, compute the condition number and how many suspect digits would be generated by this matrix. 10.15 (a) Determine the condition number for the following system using the row-sum norm. Do not normalize the system.

E 1 4 9 16 254 9 16 25 369 16 25 36 49 16 25 36 49 64

25 36 49 64 81

U

cha9792x_ch10_278-299.indd Page 298 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 298 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 299

Use suffi cient precision in displaying results to allow you to detect imprecision.

(b) Repeat part (a) using a 7 3 7 Hilbert matrix. (c) Repeat part (a) using a 10 3 10 Hilbert matrix. 10.25 Polynomial interpolation consists of determining the unique (n 2 1)th-order polynomial that fi ts n data points. Such polynomi- als have the general form,

f (x) 5 p1x n21 1 p2x

n22 1 p 1 pn21 x 1 pn (P10.25)

where the p’s are constant coeffi cients. A straightforward way for computing the coeffi cients is to generate n linear algebraic equations that we can solve simultaneously for the coeffi cients. Suppose that we want to determine the coeffi cients of the fourth-order polynomial f (x) 5 p1x

4 1 p2x 3 1 p3x

2 1 p4x 1 p5 that passes through the following fi ve points: (200, 0.746), (250, 0.675), (300, 0.616), (400, 0.525), and (500, 0.457). Each of these pairs can be substituted into Eq. (P10.25) to yield a system of fi ve equations with fi ve unknowns (the p’s). Use this approach to solve for the coeffi cients. In addition, determine and interpret the condition number.

10.23 Let the function be defi ned on the interval [0, 2] as follows:

f (x) 5 e ax 1 b, 0 # x # 1 cx 1 d, 1 # x # 2

f Determine the constants a, b, c, and d so that the function f satisfi es the following:

f(0) 5 f(2) 5 1. f is continuous on the entire interval. a 1 b 5 4.

Derive and solve a system of linear algebraic equations with a ma- trix form identical to Eq. (10.1). 10.24 (a) Create a 3 3 3 Hilbert matrix. This will be your matrix [A].

Multiply the matrix by the column vector {x} 5 [1, 1, 1]T. The solution of [A]{x} will be another column vector {b}. Using any numerical package and Gauss elimination, fi nd the solution to [A]{x} 5 {b}using the Hilbert matrix and the vector {b} that you calculated. Compare the result to your known {x} vector.

cha9792x_ch10_278-299.indd Page 299 24/10/13 3:49 PM F-468 cha9792x_ch10_278-299.indd Page 299 24/10/13 3:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11 C H A P T E R 11

300

Special Matrices and Gauss-Seidel

Certain matrices have a particular structure that can be exploited to develop effi cient solution schemes. The fi rst part of this chapter is devoted to two such systems: banded and symmetric matrices. Effi cient elimination methods are described for both. The second part of the chapter turns to an alternative to elimination methods, that is, approximate, iterative methods. The focus is on the Gauss-Seidel method, which employs initial guesses and then iterates to obtain refi ned estimates of the solution. The Gauss-Seidel method is particularly well suited for large numbers of equations. In these cases, elimination methods can be subject to round-off errors. Because the error of the Gauss-Seidel method is controlled by the number of iterations, round-off error is not an issue of concern with this method. However, there are certain instances where the Gauss- Seidel technique will not converge on the correct answer. These and other trade-offs between elimination and iterative methods will be discussed in subsequent pages.

11.1 SPECIAL MATRICES

As mentioned in Box PT3.1, a banded matrix is a square matrix that has all elements equal to zero, with the exception of a band centered on the main diagonal. Banded sys- tems are frequently encountered in engineering and scientifi c practice. For example, they typically occur in the solution of differential equations. In addition, other numerical methods such as cubic splines (Sec. 18.5) involve the solution of banded systems. The dimensions of a banded system can be quantifi ed by two parameters: the band- width BW and the half-bandwidth HBW (Fig. 11.1). These two values are related by BW 5 2HBW 1 1. In general, then, a banded system is one for which aij 5 0 if Z i 2 j Z . HBW. Although Gauss elimination or conventional LU decomposition can be employed to solve banded equations, they are ineffi cient, because if pivoting is unnecessary none of the elements outside the band would change from their original values of zero. Thus, unnecessary space and time would be expended on the storage and manipulation of these useless zeros. If it is known beforehand that pivoting is unnecessary, very effi cient algo- rithms can be developed that do not involve the zero elements outside the band. Because many problems involving banded systems do not require pivoting, these alternative al- gorithms, as described next, are the methods of choice.

cha9792x_ch11_300-318.indd Page 300 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 300 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.1 SPECIAL MATRICES 301

11.1.1 Tridiagonal Systems

A tridiagonal system—that is, one with a bandwidth of 3—can be expressed generally as

G f1 g1 e2 f2 g2

e3 f3 g3 . . .

. . . . . .

en21 fn21 gn21 en fn

W g x1 x2 x3 . . .

xn21 xn

w 5 g r1 r2 r3 . . .

rn21 rn

w (11.1) Notice that we have changed our notation for the coeffi cients from a’s and b’s to e’s, f’s, g’s, and r’s. This was done to avoid storing large numbers of useless zeros in the square matrix of a’s. This space-saving modifi cation is advantageous because the resulting al- gorithm requires less computer memory. Figure 11.2 shows pseudocode for an effi cient method, called the Thomas algorithm, to solve Eq. (11.1). As with conventional LU decomposition, the algorithm consists of three steps: decomposition and forward and back substitution. Thus, all the advantages of LU decomposition, such as convenient evaluation of multiple right-hand-side vectors and the matrix inverse, can be accomplished by proper application of this algorithm.

EXAMPLE 11.1 Tridiagonal Solution with the Thomas Algorithm

Problem Statement. Solve the following tridiagonal system with the Thomas algorithm.

≥ 2.04 21

21 2.04 21

21 2.04 21

21 2.04

¥ μ T1 T2 T3 T4

∂ 5 μ 40.8

0.8

0.8

200.8

HBW + 1

HBW

BW

Diagonal

FIGURE 11.1 Parameters used to quantify the dimensions of a banded system. BW and HBW designate the bandwidth and the half-bandwidth, respectively.

(a) Decomposition

DOFOR k 5 2, n ek 5 ek yfk21 fk 5 fk 2 ek ? gk21 END DO

(b) Forward substitution

DOFOR k 5 2, n rk 5 rk 2 ek ? rk21 END DO

(c) Back substitution

xn 5 rn yfn DOFOR k 5 n 21, 1, 21 xk 5 (rk 2 gk ? xk11)yfk END DO

FIGURE 11.2 Pseudocode to implement the Thomas algorithm, an LU decomposition method for tridi- agonal systems.

cha9792x_ch11_300-318.indd Page 301 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 301 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

302 SPECIAL MATRICES AND GAUSS-SEIDEL

Solution. First, the decomposition is implemented as

e2 5 21y2.04 5 20.49 f2 5 2.04 2 (20.49)(21) 5 1.550 e3 5 21y1.550 5 20.645 f3 5 2.04 2 (20.645)(21) 5 1.395 e4 5 21y1.395 5 20.717 f4 5 2.04 2 (20.717)(21) 5 1.323

Thus, the matrix has been transformed to

≥ 2.04 21

20.49 1.550 21

20.645 1.395 21

20.717 1.323

¥

and the LU decomposition is

[A] 5 [L][U] 5 ≥ 1

20.49 1

20.645 1

20.717 1

¥ ≥ 2.04 21

1.550 21

1.395 21

1.323

¥

You can verify that this is correct by multiplying [L][U] to yield [A]. The forward substitution is implemented as

r2 5 0.8 2 (20.49)40.8 5 20.8 r3 5 0.8 2 (20.645)20.8 5 14.221 r4 5 200.8 2 (20.717)14.221 5 210.996

Thus, the right-hand-side vector has been modifi ed to

μ 40.8

20.8

14.221

210.996

which then can be used in conjunction with the [U] matrix to perform back substitution and obtain the solution

T4 5 210.996y1.323 5 159.480 T3 5 [14.221 2 (21)159.48]y1.395 5 124.538 T2 5 [20.800 2 (21)124.538]y1.550 5 93.778 T1 5 [40.800 2 (21)93.778]y2.040 5 65.970

11.1.2 Cholesky Decomposition

Recall from Box PT3.1 that a symmetric matrix is one where aij 5 aji for all i and j. In other words, [A] 5 [A]T. Such systems occur commonly in both mathematical and

cha9792x_ch11_300-318.indd Page 302 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 302 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.1 SPECIAL MATRICES 303

engineering problem contexts. They offer computational advantages because only half the storage is needed and, in most cases, only half the computation time is required for their solution. One of the most popular approaches involves Cholesky decomposition. This algo- rithm is based on the fact that a symmetric matrix can be decomposed, as in

[A] 5 [L][L]T (11.2)

That is, the resulting triangular factors are the transpose of each other. The terms of Eq. (11.2) can be multiplied out and set equal to each other. The result can be expressed simply by recurrence relations. For the kth row,

lki 5

aki 2 a i21

j51 lij lkj

lii   for i 5 1, 2, p , k 2 1 (11.3)

and

lkk 5 B akk 2 a

k21

j51 l2kj (11.4)

EXAMPLE 11.2 Cholesky Decomposition

Problem Statement. Apply Cholesky decomposition to the symmetric matrix

[A] 5 £ 6 15 55

15 55 225

55 225 979

§

Solution. For the fi rst row (k 5 1), Eq. (11.3) is skipped and Eq. (11.4) is employed to compute

l11 5 1a11 5 16 5 2.4495

For the second row (k 5 2), Eq. (11.3) gives

l21 5 a21 l11

5 15

2.4495 5 6.1237

and Eq. (11.4) yields

l22 5 2a22 2 l 2 21 5 255 2 (6.1237)

2 5 4.1833

For the third row (k 5 3), Eq. (11.3) gives (i 5 1)

l31 5 a31 l11

5 55

2.4495 5 22.454

and (i 5 2)

l32 5 a32 2 l21l31

l22 5

225 2 6.1237(22.454)

4.1833 5 20.917

cha9792x_ch11_300-318.indd Page 303 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 303 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

304 SPECIAL MATRICES AND GAUSS-SEIDEL

Figure 11.3 presents pseudocode for implementing the Cholesky decomposition al- gorithm. It should be noted that the algorithm in Fig. 11.3 could result in an execution error if the evaluation of akk involves taking the square root of a negative number. How- ever, for cases where the matrix is positive defi nite,1 this will never occur. Because many symmetric matrices dealt with in engineering are, in fact, positive defi nite, the Cholesky algorithm has wide application. Another benefi t of dealing with positive defi nite sym- metric matrices is that pivoting is not required to avoid division by zero. Thus, we can implement the algorithm in Fig. 11.3 without the complication of pivoting.

11.2 GAUSS-SEIDEL

Iterative or approximate methods provide an alternative to the elimination methods de- scribed to this point. Such approaches are similar to the techniques we developed to obtain the roots of a single equation in Chap. 6. Those approaches consisted of guessing a value and then using a systematic method to obtain a refi ned estimate of the root. Because the present part of the book deals with a similar problem—obtaining the values that simultaneously satisfy a set of equations—we might suspect that such approximate methods could be useful in this context. The Gauss-Seidel method is the most commonly used iterative method. Assume that we are given a set of n equations:

[A]{X} 5 {B}

Suppose that for conciseness we limit ourselves to a 3 3 3 set of equations. If the di- agonal elements are all nonzero, the fi rst equation can be solved for x1, the second for x2, and the third for x3 to yield

x1 5 b1 2 a12x2 2 a13x3

a11 (11.5a)

1A positive defi nite matrix is one for which the product {X}T [A]{X} is greater than zero for all nonzero vectors {X}.

and Eq. (11.4) yields

l33 5 2a33 2 l 2 31 2 l

2 32 5 2979 2 (22.454)

2 2 (20.917)2 5 6.1101

Thus, the Cholesky decomposition yields

[L] 5 £ 2.4495

6.1237 4.1833

22.454 20.917 6.1101

§

The validity of this decomposition can be verifi ed by substituting it and its transpose into Eq. (11.2) to see if their product yields the original matrix [A]. This is left for an exercise.

FIGURE 11.3 Pseudocode for Cholesky’s LU decomposition algorithm.

DOFOR k 5 1, n DOFOR i 5 1, k 2 1 sum 5 0. DOFOR j 5 1, i 2 1 sum 5 sum 1 aij ? akj END DO aki 5 (aki 2 sum)yaii END DO sum 5 0. DOFOR j 5 1, k 2 1 sum 5 sum 1 a2kj END DO akk 5 1akk 2 sum END DO

cha9792x_ch11_300-318.indd Page 304 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 304 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.2 GAUSS-SEIDEL 305

x2 5 b2 2 a21x1 2 a23x3

a22 (11.5b)

x3 5 b3 2 a31x1 2 a32x2

a33 (11.5c)

Now, we can start the solution process by choosing guesses for the x’s. A simple way to obtain initial guesses is to assume that they are all zero. These zeros can be substituted into Eq. (11.5a), which can be used to calculate a new value for x1 5 b1ya11. Then, we substitute this new value of x1 along with the previous guess of zero for x3 into Eq. (11.5b) to compute a new value for x2. The process is repeated for Eq. (11.5c) to calculate a new estimate for x3. Then we return to the fi rst equation and repeat the entire procedure until our solution converges closely enough to the true values. Conver- gence can be checked using the criterion [recall Eq. (3.5)]

Zea,i Z 5 ` x  j i 2 x 

j21 i

x ji ` 100% , es (11.6)

for all i, where j and j 2 1 are the present and previous iterations.

EXAMPLE 11.3 Gauss-Seidel Method

Problem Statement. Use the Gauss-Seidel method to obtain the solution of the same system used in Example 10.2:

3x1 2 0.1x2 2 0.2x3 5  7.85

0.1x1 1  7x2 2 0.3x3 5 219.3

0.3x1 2 0.2x2 1 10x3 5  71.4

Recall that the true solution is x1 5 3, x2 5 22.5, and x3 5 7.

Solution. First, solve each of the equations for its unknown on the diagonal.

x1 5 7.85 1 0.1x2 1 0.2x3

3 (E11.3.1)

x2 5 219.3 2 0.1x1 1 0.3x3

7 (E11.3.2)

x3 5 71.4 2 0.3x1 1 0.2x2

10 (E11.3.3)

By assuming that x2 and x3 are zero, Eq. (E11.3.1) can be used to compute

x1 5 7.85 1 0 1 0

3 5 2.616667

This value, along with the assumed value of x3 5 0, can be substituted into Eq. (E11.3.2) to calculate

x2 5 219.3 2 0.1(2.616667) 1 0

7 5 22.794524

cha9792x_ch11_300-318.indd Page 305 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 305 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

306 SPECIAL MATRICES AND GAUSS-SEIDEL

The fi rst iteration is completed by substituting the calculated values for x1 and x2 into Eq. (E11.3.3) to yield

x3 5 71.4 2 0.3(2.616667) 1 0.2(22.794524)

10 5 7.005610

For the second iteration, the same process is repeated to compute

x1 5 7.85 1 0.1(22.794524) 1 0.2(7.005610)

3 5 2.990557   Zet Z 5 0.31%

x2 5 219.3 2 0.1(2.990557) 1 0.3(7.005610)

7 5 22.499625   Zet Z 5 0.015%

x3 5 71.4 2 0.3(2.990557) 1 0.2(22.499625)

10 5 7.000291    Zet Z 5 0.0042%

The method is, therefore, converging on the true solution. Additional iterations could be applied to improve the answers. However, in an actual problem, we would not know the true answer a priori. Consequently, Eq. (11.6) provides a means to estimate the error. For example, for x1,

Zea, 1 Z 5 ` 2.990557 2 2.6166672.990557 ` 100% 5 12.5% For x2 and x3, the error estimates are Z ea, 2 Z 5 11.8% and Z ea, 3 Z 5 0.076%. Note that, as was the case when determining roots of a single equation, formulations such as Eq. (11.6) usually provide a conservative appraisal of convergence. Thus, when they are met, they ensure that the result is known to at least the tolerance specifi ed by es.

As each new x value is computed for the Gauss-Seidel method, it is immediately used in the next equation to determine another x value. Thus, if the solution is converg- ing, the best available estimates will be employed. An alternative approach, called Jacobi iteration, utilizes a somewhat different tactic. Rather than using the latest available x’s, this technique uses Eq. (11.5) to compute a set of new x’s on the basis of a set of old x’s. Thus, as new values are generated, they are not immediately used but rather are retained for the next iteration. The difference between the Gauss-Seidel method and Jacobi iteration is depicted in Fig. 11.4. Although there are certain cases where the Jacobi method is useful, Gauss- Seidel’s utilization of the best available estimates usually makes it the method of preference.

11.2.1 Convergence Criterion for the Gauss-Seidel Method

Note that the Gauss-Seidel method is similar in spirit to the technique of simple fi xed- point iteration that was used in Sec. 6.1 to solve for the roots of a single equation. Recall that simple fi xed-point iteration had two fundamental problems: (1) it was some- times nonconvergent and (2) when it converged, it often did so very slowly. The Gauss- Seidel method can also exhibit these shortcomings.

cha9792x_ch11_300-318.indd Page 306 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 306 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.2 GAUSS-SEIDEL 307

Convergence criteria can be developed by recalling from Sec. 6.5.1 that suffi cient conditions for convergence of two nonlinear equations, u(x, y) and y(x, y), are

` 0u 0x ` 1 ` 0u

0y ` , 1 (11.7a)

and

` 0y 0x ` 1 ` 0y

0y ` , 1 (11.7b)

These criteria also apply to linear equations of the sort we are solving with the Gauss-Seidel method. For example, in the case of two simultaneous equations, the Gauss- Seidel algorithm [Eq. (11.5)] can be expressed as

u(x1, x2) 5 b1 a11

2 a12 a11

x2 (11.8a)

and

y(x1, x2) 5 b2 a22

2 a21 a22

x1 (11.8b)

The partial derivatives of these equations can be evaluated with respect to each of the unknowns as

0u 0x1

5 0   0u 0x2

5 2 a12 a11

FIGURE 11.4 Graphical depiction of the difference between (a) the Gauss-Seidel and (b) the Jacobi iterative methods for solving simultaneous linear algebraic equations.

First Iteration

x1 5 (b1 2 a12x2 2 a13x3)ya11 x1 5 (b1 2 a12x2 2 a13x3)ya11

x2 5 (b2 2 a21x1 2 a23x3)ya22 x2 5 (b2 2 a21x1 2 a23x3)ya22

x3 5 (b3 2 a31x1 2 a32x2)ya33 x3 5 (b3 2 a31x1 2 a32x2)ya33

Second Interation

x1 5 (b1 2 a12x2 2 a13x3)ya11 x1 5 (b1 2 a12x2 2 a13x3)ya11

x2 5 (b2 2 a21x1 2 a23x3)ya22 x2 5 (b2 2 a21x1 2 a23x3)ya22

x3 5 (b3 2 a31x1 2 a32x2)ya33 x3 5 (b3 2 a31x1 2 a32x2)ya33

(a) (b)

T

T

T

T

T

T ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭

cha9792x_ch11_300-318.indd Page 307 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 307 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

308 SPECIAL MATRICES AND GAUSS-SEIDEL

and

0y 0x1

5 2 a21 a22  

0y 0x2

5 0

which can be substituted into Eq. (11.7) to give

` a12 a11 ` , 1 (11.9a)

and

` a21 a22 ` , 1 (11.9b)

In other words, the absolute values of the slopes of Eq. (11.8) must be less than unity to ensure convergence. This is displayed graphically in Fig. 11.5. Equation (11.9) can also be reformulated as

Za11 Z . Za12 Z

and

Za22 Z . Za21 Z

That is, the diagonal element must be greater than the off-diagonal element for each row. The extension of the above to n equations is straightforward and can be expressed as

Zaii Z . a n

j51 j? i

Zaij Z (11.10)

FIGURE 11.5 Iteration cobwebs illustrating (a) convergence and (b) divergence of the Gauss-Seidel method. Notice that the same functions are plotted in both cases (u: 11x1 1 13x2 5 286; v: 11x1 2 9x2 5 99). Thus, the order in which the equations are implemented (as depicted by the direction of the fi rst arrow from the origin) dictates whether the computation converges.

x2

x1

v

u

(a)

x2

x1

v

u

(b)

cha9792x_ch11_300-318.indd Page 308 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 308 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.2 GAUSS-SEIDEL 309

That is, the diagonal coeffi cient in each of the equations must be larger than the sum of the absolute values of the other coeffi cients in the equation. This criterion is suffi cient but not necessary for convergence. That is, although the method may sometimes work if Eq. (11.10) is not met, convergence is guaranteed if the condition is satisfi ed. Systems where Eq. (11.10) holds are called diagonally dominant. Fortunately, many engineering problems of practical importance fulfi ll this requirement.

11.2.2 Improvement of Convergence Using Relaxation

Relaxation represents a slight modifi cation of the Gauss-Seidel method and is designed to enhance convergence. After each new value of x is computed using Eq. (11.5), that value is modifi ed by a weighted average of the results of the previous and the present iterations:

xnewi 5 lx new i 1 (1 2 l)x

old i (11.11)

where l is a weighting factor that is assigned a value between 0 and 2. If l 5 1, (1 2 l) is equal to 0 and the result is unmodifi ed. However, if l is set at a value between 0 and 1, the result is a weighted average of the present and the previous results. This type of modifi cation is called underrelaxation. It is typically employed to make a nonconvergent system converge or to hasten convergence by dampening out oscillations. For values of l from 1 to 2, extra weight is placed on the present value. In this instance, there is an implicit assumption that the new value is moving in the correct direction toward the true solution but at too slow a rate. Thus, the added weight of l is intended to improve the estimate by pushing it closer to the truth. Hence, this type of modifi cation, which is called overrelaxation, is designed to accelerate the convergence of an already convergent system. The approach is also called successive or simultaneous overrelaxation, or SOR. The choice of a proper value for l is highly problem-specifi c and is often determined empirically. For a single solution of a set of equations it is often unnecessary. However, if the system under study is to be solved repeatedly, the effi ciency introduced by a wise choice of l can be extremely important. Good examples are the very large systems of partial differential equations that often arise when modeling continuous variations of variables (recall the distributed system depicted in Fig. PT3.1b). We will return to this topic in Part Eight.

11.2.3 Algorithm for Gauss-Seidel

An algorithm for the Gauss-Seidel method, with relaxation, is depicted in Fig. 11.6. Note that this algorithm is not guaranteed to converge if the equations are not input in a diagonally dominant form. The pseudocode has two features that bear mentioning. First, there is an initial set of nested loops to divide each equation by its diagonal element. This reduces the total num- ber of operations in the algorithm. Second, notice that the error check is designated by a variable called sentinel. If any of the equations has an approximate error greater than the stopping criterion (es), then the iterations are allowed to continue. The use of the sentinel

cha9792x_ch11_300-318.indd Page 309 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 309 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

310 SPECIAL MATRICES AND GAUSS-SEIDEL

allows us to circumvent unnecessary calculations of error estimates once one of the equa- tions exceeds the criterion.

11.2.4 Problem Contexts for the Gauss-Seidel Method

Aside from circumventing the round-off dilemma, the Gauss-Seidel technique has a num- ber of other advantages that make it particularly attractive in the context of certain en- gineering problems. For example, when the matrix in question is very large and very sparse (that is, most of the elements are zero), elimination methods waste large amounts of computer memory by storing zeros.

FIGURE 11.6 Pseudocode for Gauss-Seidel with relaxation.

SUBROUTINE Gseid (a,b,n,x,imax,es,lambda) DOFOR i 5 1,n dummy 5 ai,i DOFOR j 5 1,n ai,j 5 ai,j/dummy END DO bi 5 bi/dummy END DO DOFOR i 5 1, n sum 5 bi DOFOR j 5 1, n IF i fi j THEN sum 5 sum 2 ai,j*xj END DO xi5sum END DO iter51 DO sentinel 5 1 DOFOR i 5 1,n old 5 xi sum 5 bi DOFOR j 5 1,n IF i fi j THEN sum 5 sum 2 ai,j*xj END DO xi 5 lambda*sum 1(1.2lambda)*old IF sentinel 5 1 AND xi fi 0. THEN ea 5 ABS((xi 2 old)/xi)*100. IF ea . es THEN sentinel 5 0 END IF END DO iter 5 iter 1 1 IF sentinel 5 1 OR (iter $ imax) EXIT END DO END Gseid

cha9792x_ch11_300-318.indd Page 310 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 310 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.3 LINEAR ALGEBRAIC EQUATIONS WITH SOFTWARE PACKAGES 311

At the beginning of this chapter, we saw how this shortcoming could be circum- vented if the coeffi cient matrix is banded. For nonbanded systems, there is usually no simple way to avoid large memory requirements when using elimination methods. Be- cause all computers have a fi nite amount of memory, this ineffi ciency can place a con- straint on the size of systems for which elimination methods are practical. Although a general algorithm such as the one in Fig. 11.6 is prone to the same constraint, the structure of the Gauss-Seidel equations [Eq. (11.5)] permits concise pro- grams to be developed for specifi c systems. Because only nonzero coeffi cients need be included in Eq. (11.5), large savings of computer memory are possible. Although this entails more up-front investment in software development, the long-term advantages are substantial when dealing with large systems for which many simulations are to be per- formed. Both lumped- and distributed-variable systems can result in large, sparse matri- ces for which the Gauss-Seidel method has utility.

11.3 LINEAR ALGEBRAIC EQUATIONS WITH SOFTWARE PACKAGES

Software packages have great capabilities for solving systems of linear algebraic equa- tions. Before describing these tools, we should mention that the approaches described in Chap. 7 for solving nonlinear systems can be applied to linear systems. However, in this section, we will focus on the approaches that are expressly designed for linear equations.

11.3.1 Excel

There are two ways to solve linear algebraic equations with Excel: (1) using the Solver tool or (2) using matrix inversion and multiplication functions. Recall that one way to determine the solution of linear algebraic equations is

{X} 5 [A]21{B} (11.12)

Excel has built-in functions for both matrix inversion and multiplication that can be used to implement this formula.

EXAMPLE 11.4 Using Excel to Solve Linear Systems

Problem Statement. Recall that in Chap. 10 we introduced the Hilbert matrix. The following system is based on the Hilbert matrix. Note that it is scaled, as was done previously in Example 10.3, so that the maximum coeffi cient in each row is unity.

£ 1 1y2 1y3 1 2y3 1y2 1 3y4 3y5

§ • x1 x2 x3

¶ 5 • 1.833333

2.166667

2.35

The solution to this system is {X}T 5 :1 1 1 ; . Use Excel to obtain this solution. Solution. The spreadsheet to solve this problem is displayed in Fig. 11.7. First, the matrix [A] and the right-hand-side constants {B} are entered into the spreadsheet cells. Then, a set of cells of the proper dimensions (in our example 3 3 3) is highlighted by either clicking and dragging the mouse or by using the arrow keys while depressing the shift key. As in Fig. 11.7, we highlight the range: B5. .D7.

S O F T W A R E

cha9792x_ch11_300-318.indd Page 311 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 311 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

312 SPECIAL MATRICES AND GAUSS-SEIDEL

Next, a formula invoking the matrix inverse function is entered,

=minverse(B1..D3)

Note that the argument is the range holding the elements of [A]. The Ctrl and Shift keys are held down while the Enter key is depressed. The resulting inverse of [A] will be calculated by Excel and displayed in the range B5. .D7 as shown in Fig. 11.7. A similar approach is used to multiply the inverse by the right-hand-side vector. For this case, the range from F5. .F7 is highlighted and the following formula is entered

=mmult(B5..D7,F1..F3)

where the fi rst range is the fi rst matrix to be multiplied, [A]21, and the second range is the second matrix to be multiplied, {B}. By again using the Ctrl-Shift-Enter combination, the solution {X} will be calculated by Excel and displayed in the range F5. .F7, as shown in Fig. 11.7. As can be seen, the correct answer results.

FIGURE 11.7

Notice that we deliberately reformatted the results in Example 11.4 to show 15 digits. We did this because Excel uses double-precision to store numerical values. Thus, we see that round-off error occurs in the last two digits. This implies a condition number on the order of 100, which agrees with the result of 451.2 originally calculated in Example 10.3. Excel does not have the capability to calculate a condition number. In most cases, particularly because it employs double-precision numbers, this does not rep- resent a problem. However, for cases where you suspect that the system is ill-conditioned, determination of the condition number is useful. MATLAB and Mathcad software are capable of computing this quantity.

11.3.2 MATLAB

As the name implies, MATLAB (short for MATrix LABoratory) was designed to facili- tate matrix manipulations. Thus, as might be expected, its capabilities in this area are

S O F T W A R E

cha9792x_ch11_300-318.indd Page 312 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 312 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.3 LINEAR ALGEBRAIC EQUATIONS WITH SOFTWARE PACKAGES 313

excellent. Some of the key MATLAB functions related to matrix operations are listed in Table 11.1. The following example illustrates a few of these capabilities.

EXAMPLE 11.5 Using MATLAB to Manipulate Linear Algebraic Equations

Problem Statement. Explore how MATLAB can be employed to solve and analyze linear algebraic equations. Use the same system as in Example 11.4.

Solution. First, we can enter the [A] matrix and the {B}vector,

>> A 5 [ 1 1/2 1/3 ; 1 2/3 1/2 ; 1 3/4 3/5 ]

A = 1.0000 0.5000 0.3333 1.0000 0.6667 0.5000 1.0000 0.7500 0.6000

>> B=[1+1/2+1/3;1+2/3+2/4;1+3/4+3/5]

B = 1.8333 2.1667 2.3500

Next, we can determine the condition number for [A], as in

>> cond(A)

ans = 366.3503

TABLE 11.1 MATLAB functions to implement matrix analysis and numerical linear algebra.

Matrix Analysis Linear Equations

Function Description Function Description

cond Matrix condition number \ and / Linear equation solution; use “help slash” norm Matrix or vector norm chol Cholesky factorization rcond LINPACK reciprocal condition estimator lu Factors from Gauss elimination rank Number of linearly independent inv Matrix inverse rows or columns det Determinant qr Orthogonal-triangular decomposition trace Sum of diagonal elements qrdelete Delete a column from the QR factorization null Null space qrinsert Insert a column in the QR factorization orth Orthogonalization nnls Nonnegative least squares rref Reduced row echelon form pinv Pseudoinverse lscov Least squares in the presence of known

covariance

cha9792x_ch11_300-318.indd Page 313 24/10/13 8:05 PM F-468 cha9792x_ch11_300-318.indd Page 313 24/10/13 8:05 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

314 SPECIAL MATRICES AND GAUSS-SEIDEL

This result is based on the spectral, or B AB2, norm discussed in Box 10.2. Note that it is of the same order of magnitude as the condition number 5 451.2 based on the row- sum norm in Example 10.3. Both results imply that between two and three digits of precision could be lost. Now we can solve the system of equations in two different ways. The most direct and effi cient way is to employ backslash, or “left division”:

>> X=A\B

X = 1.0000 1.0000 1.0000

For cases such as ours, MATLAB uses Gauss elimination to solve such systems. As an alternative, we can implement Eq. (PT3.6) directly, as in

>> X=inv(A)*B

X = 1.0000 1.0000 1.0000

This approach actually determines the matrix inverse fi rst and then performs the matrix multiplication. Hence, it is more time consuming than using the backslash approach.

S O F T W A R E

11.3.3 Mathcad

Mathcad contains many special functions that manipulate vectors and matrices. These include common operations such as the dot product, matrix transpose, matrix addition, and matrix multiplication. In addition, it allows calculation of the matrix inverse, deter- minant, trace, various types of norms, and condition numbers based on different norms. It also has several functions that decompose matrices. Systems of linear equations can be solved in two ways by Mathcad. First, it is pos- sible to use matrix inversion and subsequent multiplication by the right-hand-side as discussed in Chap. 10. In addition, Mathcad has a special function called lsolve(A,b) that is specifi cally designed to solve linear equations. You can use other built-in functions to evaluate the condition of A to determine if A is nearly singular and thus possibly subject to round-off errors. As an example, let’s use lsolve to solve a system of linear equations. As shown in Fig. 11.8, the fi rst step is to enter the coeffi cients of the A matrix using the defi nition symbol and the Insert/Matrix pull down menu. This gives a box that allows you to

cha9792x_ch11_300-318.indd Page 314 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 314 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

11.3 LINEAR ALGEBRAIC EQUATIONS WITH SOFTWARE PACKAGES 315

specify the dimensions of the matrix. For our case, we will select a dimension of 434, and Mathcad places a blank 4-by-4-size matrix on screen. Now, simply click the a ppropriate cell location and enter values. Repeat similar operations to create the right- hand-side b vector. Now the vector x is defi ned as lsolve(A,b) and the value of x is displayed with the equal sign. We can also solve the same system using the matrix inverse. The inverse can be simply computed by merely raising A to the exponent 21. The result is shown on the right side of Fig. 11.8. The solution is then generated as the product of the inverse times b. Next, let’s use Mathcad to fi nd the inverse and the condition number of the Hilbert matrix. As in Fig. 11.9, the scaled matrix can be entered using the defi nition symbol and the Insert/Matrix pull down menu. The inverse can again be computed by simply raising H to the exponent 21. The result is shown in Fig. 11.9. We can then use some other Mathcad functions to determine condition numbers by using the defi nition symbol to defi ne variables c1, c2, ce, and ci as the condition number based on the column-sum (cond1), spectral (cond2), the Euclidean (conde), and the row-sum (condi) norms, re- spectively. The resulting values are shown at the bottom of Fig. 11.9. As expected, the spectral norm provides the smallest measure of magnitude.

FIGURE 11.8 Mathcad screen to solve a system of linear algebraic equations.

cha9792x_ch11_300-318.indd Page 315 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 315 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

316 SPECIAL MATRICES AND GAUSS-SEIDEL

FIGURE 11.9 Mathcad screen to determine the matrix inverse and condition numbers of a scaled 333 Hilbert matrix.

S O F T W A R E

PROBLEMS

11.1 Perform the same calculations as in (a) Example 11.1, and (b) Example 11.3, but for the tridiagonal system,

£ 0.8 20.4

20.4 0.8 20.4

20.4 0.8

§ • x1 x2 x3

¶ 5 • 41

25

105

11.2 Determine the matrix inverse for Example 11.1 based on the LU decomposition and unit vectors. 11.3 The following tridiagonal system must be solved as part of a larger algorithm (Crank-Nicolson) for solving partial differential equations:

D 2.01475 20.02087520.020875 2.01475 20.020875 20.020875 2.01475 20.020875

20.020875 2.01475

T 3 dT1T2

T3 T4

t 5 d 4.1750 0

2.0875

t Use the Thomas algorithm to obtain a solution. 11.4 Confirm the validity of the Cholesky decomposition of Example 11.2 by substituting the results into Eq. (11.2) to see if the product of [L] and [L]T yields [A].

cha9792x_ch11_300-318.indd Page 316 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 316 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 317

11.13 Use the Gauss-Seidel method (a) without relaxation and (b) with relaxation (l 5 1.2) to solve the following system to a tolerance of es 5 5%. If necessary, rearrange the equations to achieve convergence.

2x1 2 6x2 2 x3 5 238

23x1 2 x2 1 7x3 5 234

28x1 1 x2 2 2x3 5 220

11.14 Redraw Fig. 11.5 for the case where the slopes of the equa- tions are 1 and 21. What is the result of applying Gauss-Seidel to such a system? 11.15 Of the following three sets of linear equations, identify the set(s) that you could not solve using an iterative method such as Gauss-Seidel. Show using any number of iterations that is neces- sary that your solution does not converge. Clearly state your con- vergence criteria (how you know it is not converging).

Set One Set Two Set Three

8x 1 3y 1 z 5 12 x 1 y 1 5z 5 7 2x 1 3y 1 5z 5 7 26x 1 7z 5 1 x 1 4y 2 z 5 4 22x 1 4y 2 5z 5 23 2x 1 4y 2 z 5 5 3x 1 y 2 z 5 4 2y 2 z 5 1

11.16 Use the software package of your choice to obtain a solu- tion, calculate the inverse, and determine the condition number (without scaling) based on the row-sum norm for (a)

£ 1 4 9

4 9 16

9 16 25

§ • x1 x2 x3

¶ 5 • 14

29

50

(b)

D 1 4 9 164 9 16 25 9 16 25 36

16 25 36 49

T dx1x2 x3 x4

t 5 d 3054 86

126

t In both cases, the answers for all the x’s should be 1. 11.17 Given the pair of nonlinear simultaneous equations:

f(x, y) 5 4 2 y 2 2x2

g(x, y) 5 8 2 y2 2 4x

(a) Use the Excel Solver to determine the two pairs of values of x and y that satisfy these equations.

(b) Using a range of initial guesses (x 5 26 to 6 and y 5 26 to 6), determine which initial guesses yield each of the solutions.

11.5 Perform the same calculations as in Example 11.2, but for the symmetric system,

£ 6 15 55

15 55 225

55 225 979

§ • a0 a1 a2

¶ 5 • 152.6

585.6

2488.8

In addition to solving for the Cholesky decomposition, employ it to solve for the a’s. 11.6 Perform a Cholesky decomposition of the following symmet- ric system by hand,

£ 8 20 15

20 80 50

15 50 60

§ • x1 x2 x3

¶ 5 • 50

250

100

11.7 Compute the Cholesky decomposition of

[A] 5 £ 9 0 0

0 25 0

0 0 4

§

Do your results make sense in terms of Eqs. (11.3) and (11.4)? 11.8 Use the Gauss-Seidel method to solve the tridiagonal system from Prob. 11.1 (es 5 5%). Use overrelaxation with l 5 1.2. 11.9 Recall from Prob. 10.8, that the following system of equa- tions is designed to determine concentrations (the c’s in g/m3) in a series of coupled reactors as a function of amount of mass input to each reactor (the right-hand sides in g/d),

15c1 2 3c2 2 c3 5 3800

23c1 1 18c2 2 6c3 5 1200

24c1 2 c2 1 12c3 5 2350

Solve this problem with the Gauss-Seidel method to es 5 5%. 11.10 Repeat Prob. 11.9, but use Jacobi iteration. 11.11 Use the Gauss-Seidel method to solve the following system until the percent relative error falls below es 5 5%,

10x1 1 2x2 2 x3 5 27

23x1 2 6x2 1 2x3 5 261.5

x1 1 x2 1 5x3 5 221.5

11.12 Use the Gauss-Seidel method (a) without relaxation and (b) with relaxation (l 5 0.95) to solve the following system to a tolerance of es 5 5%. If necessary, rearrange the equations to achieve convergence.

23x1 1 x2 1 12x3 5 50

6x1 2 x2 2 x3 5 3

6x1 1 9x2 1 x3 5 40

cha9792x_ch11_300-318.indd Page 317 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 317 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

318 SPECIAL MATRICES AND GAUSS-SEIDEL

11.24 Develop a user-friendly program in either a high-level or macro language of your choice to obtain a solution for a tridiagonal system with the Thomas algorithm (Fig. 11.2). Test your program by duplicating the results of Example 11.1. 11.25 Develop a user-friendly program in either a high-level or macro language of your choice for Cholesky decomposition based on Fig. 11.3. Test your program by duplicating the results of Example 11.2. 11.26 Develop a user-friendly program in either a high-level or macro language of your choice for the Gauss-Seidel method based on Fig. 11.6. Test your program by duplicating the results of Example 11.3. 11.27 As described in Sec. PT3.1.2, linear algebraic equations can arise in the solution of differential equations. For example, the following differential equation results from a steady-state mass balance for a chemical in a one-dimensional canal,

0 5 D d2c

dx2 2 U

dc

dx 2 kc

where c 5 concentration, t 5 time, x 5 distance, D 5 diffusion coeffi cient, U 5 fl uid velocity, and k 5 a fi rst-order decay rate. Convert this differential equation to an equivalent system of simul- taneous algebraic equations. Given D 5 2, U 5 1, k 5 0.2, c(0) 5 80 and c(10) 5 20, solve these equations from x 5 0 to 10 with Dx 5 2, and develop a plot of concentration versus distance. 11.28 A pentadiagonal system with a bandwidth of fi ve can be expressed generally as

Develop a program to effi ciently solve such systems without pivoting in a similar fashion to the algorithm used for tridiagonal matrices in Sec. 11.1.1. Test it for the following case:

E 8 22 21 0 022 9 24 21 021 23 7 21 22 0 24 22 12 25

0 0 27 23 15

U ex1x2x3 x4 x5

u 5 e520 1

5

u

H f1 g1 h1 e2 f2 g2 h2 d3 e3 f3 g3 h3

. . .

. . .

. . .

dn21 en21 fn21 gn21 dn en fn

X h x1 x2 x3 .

.

.

xn21 xn

x5h r1 r2 r3 .

.

.

rn21 rn

x

11.18 An electronics company produces transistors, resistors, and computer chips. Each transistor requires four units of copper, one unit of zinc, and two units of glass. Each resistor requires three, three, and one units of the three materials, respectively, and each computer chip requires two, one, and three units of these materials, respectively. Putting this information into table form, we get:

Component Copper Zinc Glass

Transistors 4 1 2 Resistors 3 3 1 Computer chips 2 1 3

Supplies of these materials vary from week to week, so the com- pany needs to determine a different production run each week. For example, one week the total amounts of materials available are 960 units of copper, 510 units of zinc, and 610 units of glass. Set up the system of equations modeling the production run, and use Excel, MATLAB, or Mathcad, to solve for the number of transistors, resis- tors, and computer chips to be manufactured this week. 11.19 Use MATLAB or Mathcad software to determine the spectral condition number for a 10-dimensional Hilbert matrix. How many digits of precision are expected to be lost due to ill-conditioning? Determine the solution for this system for the case where each ele- ment of the right-hand-side vector {b} consists of the summation of the coeffi cients in its row. In other words, solve for the case where all the unknowns should be exactly one. Compare the resulting er- rors with those expected based on the condition number. 11.20 Repeat Prob. 11.19, but for the case of a six-dimensional Vandermonde matrix (see Prob. 10.17) where x1 5 4, x2 5 2, x3 5 7, x4 5 10, x5 5 3, and x6 5 5. 11.21 Given a square matrix [A], write a single line MATLAB command that will create a new matrix [Aug] that consists of the original matrix [A] augmented by an identity matrix [I]. 11.22 Write the following set of equations in matrix form:

50 5 5x3 2 7x2

4x2 1 7x3 1 30 5 0

x1 2 7x3 5 40 2 3x2 1 5x1

Use Excel, MATLAB, or Mathcad to solve for the unknowns. In addition, compute the transpose and the inverse of the coeffi cient matrix. 11.23 In Sec. 9.2.1, we determined the number of operations re- quired for Gauss elimination without partial pivoting. Make a simi- lar determination for the Thomas algorithm (Fig. 11.2). Develop a plot of operations versus n (from 2 to 20) for both techniques.

cha9792x_ch11_300-318.indd Page 318 23/10/13 5:40 PM F-468 cha9792x_ch11_300-318.indd Page 318 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12 C H A P T E R 12

319

Case Studies: Linear Algebraic Equations

The purpose of this chapter is to use the numerical procedures discussed in Chaps. 9, 10, and 11 to solve systems of linear algebraic equations for some engineering case studies. These systematic numerical techniques have practical signifi cance because engineers fre- quently encounter problems involving systems of equations that are too large to solve by hand. The numerical algorithms in these applications are particularly convenient to imple- ment on personal computers. Section 12.1 shows how a mass balance can be employed to model a system of reactors. Section 12.2 places special emphasis on the use of the matrix inverse to determine the complex cause-effect interactions between forces in the members of a truss. Section 12.3 is an example of the use of Kirchhoff’s laws to compute the cur- rents and voltages in a resistor circuit. Finally, Sec. 12.4 is an illustration of how linear equations are employed to determine the steady-state confi guration of a mass- spring system.

12.1 STEADY-STATE ANALYSIS OF A SYSTEM OF REACTORS (CHEMICAL/BIO ENGINEERING)

Background. One of the most important organizing principles in chemical engineer- ing is the conservation of mass (recall Table 1.1). In quantitative terms, the principle is expressed as a mass balance that accounts for all sources and sinks of a material that pass in and out of a volume (Fig. 12.1). Over a fi nite period of time, this can be expressed as

Accumulation 5 inputs 2 outputs (12.1)

The mass balance represents a bookkeeping exercise for the particular substance being modeled. For the period of the computation, if the inputs are greater than the outputs, the mass of the substance within the volume increases. If the outputs are greater than the inputs, the mass decreases. If inputs are equal to the outputs, accumulation is zero and mass remains constant. For this stable condition, or steady state, Eq. (12.1) can be expressed as

Inputs 5 outputs (12.2)

cha9792x_ch12_319-340.indd Page 319 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 319 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

320 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

Employ the conservation of mass to determine the steady-state concentrations of a system of coupled reactors.

Solution. The mass balance can be used for engineering problem solving by expressing the inputs and outputs in terms of measurable variables and parameters. For example, if we were performing a mass balance for a conservative substance (that is, one that does not increase or decrease due to chemical transformations) in a reactor (Fig. 12.2), we would have to quantify the rate at which mass fl ows into the reactor through the two infl ow pipes and out of the reactor through the outfl ow pipe. This can be done by taking the product of the fl ow rate Q (in cubic meters per minute) and the concentration c (in milligrams per cubic meter) for each pipe. For example, for pipe 1 in Fig. 12.2, Q1 5 2 m3/min and c1 5 25 mg/m

3; therefore, the rate at which mass fl ows into the reactor through pipe 1 is Q1c1 5 (2 m

3/min)(25 mg/m3) 5 50 mg/min. Thus, 50 mg of chemi- cal fl ows into the reactor through this pipe each minute. Similarly, for pipe 2 the mass infl ow rate can be calculated as Q2c2 5 (1.5 m

3/min)(10 mg/m3) 5 15 mg/min. Notice that the concentration out of the reactor through pipe 3 is not specifi ed by Fig. 12.2. This is because we already have suffi cient information to calculate it on the basis of the conservation of mass. Because the reactor is at steady state, Eq. (12.2) holds and the inputs should be in balance with the outputs, as in

Q1c1 1 Q2c2 5 Q3c3

Substituting the given values into this equation yields

50 1 15 5 3.5c3

which can be solved for c3 5 18.6 mg/m 3. Thus, we have determined the concentration

in the third pipe. However, the computation yields an additional bonus. Because the reactor is well mixed (as represented by the propeller in Fig. 12.2), the concentration will be uniform, or homogeneous, throughout the tank. Therefore the concentration in pipe 3 should be identical to the concentration throughout the reactor. Consequently, the mass balance has allowed us to compute both the concentration in the reactor and in the

Input Output

Accumulation

Volume

FIGURE 12.1 A schematic representation of mass balance.

cha9792x_ch12_319-340.indd Page 320 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 320 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12.1 STEADY-STATE ANALYSIS OF A SYSTEM OF REACTORS 321

outfl ow pipe. Such information is of great utility to chemical and petroleum engineers who must design reactors to yield mixtures of a specifi ed concentration. Because simple algebra was used to determine the concentration for the single reac- tor in Fig. 12.2, it might not be obvious how computers fi gure in mass-balance calcula- tions. Figure 12.3 shows a problem setting where computers are not only useful but are a practical necessity. Because there are fi ve interconnected, or coupled, reactors, fi ve simultaneous mass-balance equations are needed to characterize the system. For reactor 1, the rate of mass fl ow in is

5(10) 1 Q31c3

FIGURE 12.2 A steady-state, completely mixed reactor with two infl ow pipes and one outfl ow pipe. The fl ows Q are in cubic meters per minute, and the concentra- tions c are in milligrams per cubic meter.

Q3 = 3.5 m 3/min

c3 = ?

Q1 = 2 m 3/min

c1 = 25 mg/m 3

Q2 = 1.5 m 3/min

c2 = 10 mg/m 3

Q24 = 1

Q54 = 2

Q55 = 2Q15 = 3

Q44 = 11Q12 = 3

Q31 = 1 Q03 = 8

c03 = 20

Q23 = 1

Q25 = 1

Q34 = 8

Q01 = 5

c01 = 10

c3

c5

c1 c2 c4

FIGURE 12.3 Five reactors linked by pipes.

cha9792x_ch12_319-340.indd Page 321 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 321 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

322 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

and the rate of mass fl ow out is

Q12c1 1 Q15c1

Because the system is at steady state, the infl ows and outfl ows must be equal:

5(10) 1 Q31c3 5 Q12c1 1 Q15c1

or, substituting the values for fl ow from Fig. 12.3,

6c1 2 c3 5 50

Similar equations can be developed for the other reactors:

23c1 1 3c2 5 0

2c2 1 9c3 5 160

2c2 2 8c3 1 11c4 2 2c5 5 0

23c1 2 c2 1 4c5 5 0

A numerical method can be used to solve these fi ve equations for the fi ve unknown concentrations:

{C}T 5 :11.51 11.51 19.06 17.00 11.51 ; In addition, the matrix inverse can be computed as

[A]21 5 E0.16981 0.00629 0.01887 0 00.16981 0.33962 0.01887 0 00.01887 0.03774 0.11321 0 0 0.06003 0.07461 0.08748 0.09091 0.04545

0.16981 0.08962 0.01887 0 0.25000

U Each of the elements aij signifi es the change in concentration of reactor i due to a unit change in loading to reactor j. Thus, the zeros in column 4 indicate that a loading to reactor 4 will have no impact on reactors 1, 2, 3, and 5. This is consistent with the system confi guration (Fig. 12.3), which indicates that fl ow out of reactor 4 does not feed back into any of the other reactors. In contrast, loadings to any of the fi rst three reactors will affect the entire system as indicated by the lack of zeros in the fi rst three columns. Such information is of great utility to engineers who design and manage such systems.

12.2 ANALYSIS OF A STATICALLY DETERMINATE TRUSS (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. An important problem in structural engineering is that of fi nding the forces and reactions associated with a statically determinate truss. Figure 12.4 shows an example of such a truss. The forces (F) represent either tension or compression on the members of the truss. External reactions (H2, V2, and V3) are forces that characterize how the truss interacts with the supporting surface. The hinge at node 2 can transmit both horizontal and vertical forces to the surface, whereas the roller at node 3 transmits only vertical forces. It is observed that the ef- fect of the external loading of 1000 lb is distributed among the various members of the truss.

cha9792x_ch12_319-340.indd Page 322 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 322 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12.2 ANALYSIS OF A STATICALLY DETERMINATE TRUSS 323

Solution. This type of structure can be described as a system of coupled linear alge- braic equations. Free-body force diagrams are shown for each node in Fig. 12.5. The sum of the forces in both horizontal and vertical directions must be zero at each node, because the system is at rest. Therefore, for node 1,

gFH 5 0 5 2F1 cos 30° 1 F3 cos 60° 1 F1, h (12.3) gFV 5 0 5 2F1 sin 30° 2 F3 sin 60° 1 F1, y (12.4)

for node 2,

gFH 5 0 5 F2 1 F1 cos 30° 1 F2, h 1 H2 (12.5) gFV 5 0 5 F1 sin 30° 1 F2, y 1 V2 (12.6)

FIGURE 12.4 Forces on a statically determi- nate truss.

1000 lb

2 3

1

30� 60�

90� F3F1

F2

H2

V2 V3

FIGURE 12.5 Free-body force diagrams for the nodes of a statically determinate truss.

2 F3,h

F1,v

F1,h

F2

F2,h

F1 F2,v

H2

V2

F3 F1

F3,v

F3

F2

V3

1

30�

30�

60�

60�

3

cha9792x_ch12_319-340.indd Page 323 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 323 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

324 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

for node 3,

gFH 5 0 5 2F2 2 F3 cos 60° 1 F3, h (12.7) gFV 5 0 5 F3 sin 60° 1 F3, y 1 V3 (12.8)

where Fi, h is the external horizontal force applied to node i (where a positive force is from left to right) and F1, y is the external vertical force applied to node i (where a positive force is upward). Thus, in this problem, the 1000-lb downward force on node 1 corresponds to F1, y 5 21000. For this case all other Fi, y’s and Fi, h’s are zero. Note that the directions of the internal forces and reactions are unknown. Proper application of Newton’s laws requires only consistent assumptions regarding direction. Solutions are negative if the directions are assumed incorrectly. Also note that in this problem, the forces in all members are assumed to be in tension and act to pull adjoining nodes to- gether. A negative solution therefore corresponds to compression. This problem can be written as the following system of six equations and six unknowns:

F 0.866 0 20.5 0 0 00.5 0 0.866 0 0 020.866 21 0 21 0 0 20.5 0 0 0 21 0

0 1 0.5 0 0 0

0 0 20.866 0 0 21

V fF1F2F3 H2 V2 V3

v 5 f 0210000 0

0

0

v (12.9) Notice that, as formulated in Eq. (12.9), partial pivoting is required to avoid division by zero diagonal elements. Employing a pivot strategy, the system can be solved using any of the elimination techniques discussed in Chap. 9 or 10. However, because this problem is an ideal case study for demonstrating the utility of the matrix inverse, the LU decomposition can be used to compute

F1 5 2500   F2 5 433   F3 5 2866

H2 5 0  V2 5 250  V3 5 750

and the matrix inverse is

[A]21 5 F 0.866 0.50.25 20.43320.5 0.866 21 0

20.433 20.25

0.433 20.75

0 0 0 0

0 0 1 0

0 0 0 0

21 0 21 0

0 21 0 0

0 0 0 21

V Now, realize that the right-hand-side vector represents the externally applied horizontal and vertical forces on each node, as in

{F}T 5 :F1, h F1, y F2, h F2, y F3, h F3, y ; (12.10) Because the external forces have no effect on the LU decomposition, the method need not be implemented over and over again to study the effect of different external forces on the truss. Rather, all that we have to do is perform the forward- and backward-substitution steps for each right-hand-side vector to effi ciently obtain alternative solutions. For example,

cha9792x_ch12_319-340.indd Page 324 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 324 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12.2 ANALYSIS OF A STATICALLY DETERMINATE TRUSS 325

we might want to study the effect of horizontal forces induced by a wind blowing from left to right. If the wind force can be idealized as two point forces of 1000 lb on nodes 1 and 2 (Fig. 12.6a), the right-hand-side vector is

{F}T 5 :21000 0 1000 0 0 0 ; which can be used to compute

F1 5 866   F2 5 250   F3 5 2500

H2 5 22000   V2 5 2433   V3 5 433

For a wind from the right (Fig. 12.6b), F1, h 5 21000, F3, h 5 21000, and all other external forces are zero, with the result that

F1 5 2866   F2 5 21250   F3 5 500

H2 5 2000   V2 5 433   V3 5 2433

The results indicate that the winds have markedly different effects on the structure. Both cases are depicted in Fig. 12.6. The individual elements of the inverted matrix also have direct utility in elucidating stimulus-response interactions for the structure. Each element represents the change of one of the unknown variables to a unit change of one of the external stimuli. For ex- ample, element a2132 indicates that the third unknown (F3) will change 0.866 due to a unit change of the second external stimulus (F1, y). Thus, if the vertical load at the fi rst node were increased by 1, F3 would increase by 0.866. The fact that elements are 0 indicates that certain unknowns are unaffected by some of the external stimuli. For instance a2132 5 0 means that F1 is unaffected by changes in F2, h. This ability to isolate interactions has a number of engineering applications, including the identifi cation of those compo- nents that are most sensitive to external stimuli and, as a consequence, most prone to failure. In addition, it can be used to determine components that may be unnecessary (see Prob. 12.18).

FIGURE 12.6 Two test cases showing (a) winds from the left and (b) winds from the right.

(a) (b)

86 6

2000 1000

1000

250

5 0 0

433 433

86 6

2000 1000

1000

1250 5 0 0

433 433

cha9792x_ch12_319-340.indd Page 325 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 325 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

326 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

The foregoing approach becomes particularly useful when applied to large complex structures. In engineering practice, it may be necessary to solve trusses with hundreds or even thousands of structural members. Linear equations provide one powerful ap- proach for gaining insight into the behavior of these structures.

12.3 CURRENTS AND VOLTAGES IN RESISTOR CIRCUITS (ELECTRICAL ENGINEERING)

Background. A common problem in electrical engineering involves determining the currents and voltages at various locations in resistor circuits. These problems are solved using Kirchhoff’s current and voltage rules. The current (or point) rule states that the algebraic sum of all currents entering a node must be zero (see Fig. 12.7a), or

oi 5 0 (12.11)

where all current entering the node is considered positive in sign. The current rule is an application of the principle of conservation of charge (recall Table 1.1). The voltage (or loop) rule specifi es that the algebraic sum of the potential differences (that is, voltage changes) in any loop must equal zero. For a resistor circuit, this is ex- pressed as

oj 2 oi R 5 0 (12.12)

where j is the emf (electromotive force) of the voltage sources and R is the resistance of any resistors on the loop. Note that the second term derives from Ohm’s law (Fig. 12.7b), which states that the voltage drop across an ideal resistor is equal to the product of the current and the resistance. Kirchhoff’s voltage rule is an expression of the conservation of energy.

Solution. Application of these rules results in systems of simultaneous linear algebraic equations because the various loops within a circuit are coupled. For example, consider the circuit shown in Fig. 12.8. The currents associated with this circuit are unknown both in magnitude and direction. This presents no great diffi culty because one simply assumes a direction for each current. If the resultant solution from Kirchhoff’s laws is negative, then the assumed direction was incorrect. For example, Fig. 12.9 shows some assumed currents.

FIGURE 12.7 Schematic representations of (a) Kirchhoff’s current rule and (b) Ohm’s law.

i1 i3

i2

Vi Vj Rij

iij

(a)

(b)

FIGURE 12.8 A resistor circuit to be solved using simultaneous linear algebraic equations.

R = 5 � R = 10 �

R = 10 �3 2 1

4 5 6R = 15 �

R = 5 � V1 = 200 V

V6 = 0 V R = 20 �

cha9792x_ch12_319-340.indd Page 326 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 326 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12.3 CURRENTS AND VOLTAGES IN RESISTOR CIRCUITS 327

Given these assumptions, Kirchhoff’s current rule is applied at each node to yield

i12 1 i52 1 i32 5 0

i65 2 i52 2 i54 5 0

i43 2 i32 5 0

i54 2 i43 5 0

Application of the voltage rule to each of the two loops gives

2i54 R54 2 i43 R43 2 i32 R32 1 i52 R52 5 0

2i65 R65 2 i52 R52 2 i12 R12 2 200 5 0

or, substituting the resistances from Fig. 12.8 and bringing constants to the right-hand side,

215i54 2 5i43 2 10i32 1 10i52 5 0

220i65 2 10i52 1 5i12 5 200

Therefore, the problem amounts to solving the following set of six equations with six unknown currents:

F1 1   1   0   0   00 21   0   1   21   00 0   21   0   0   10 0   0   0   1   21 0 10 210 0   215 25

5 210 0   220 0   0

V fi12i52i32i65 i54 i43

v 5 f 000 0

0

200

v Although impractical to solve by hand, this system is easily handled using an elimination method. Proceeding in this manner, the solution is

i12 5 6.1538   i52 5 24.6154   i32 5 21.5385

i65 5 26.1538   i54 5 21.5385   i43 5 21.5385

Thus, with proper interpretation of the signs of the result, the circuit currents and volt- ages are as shown in Fig. 12.10. The advantages of using numerical algorithms and computers for problems of this type should be evident.

FIGURE 12.9 Assumed currents.

3 2 1

4 5 6

i12

i65 i52

i32

i54 i43

cha9792x_ch12_319-340.indd Page 327 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 327 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

328 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

12.4 SPRING-MASS SYSTEMS (MECHANICAL/AEROSPACE ENGINEERING)

Background. Idealized spring-mass systems play an important role in mechanical and other engineering problems. Figure 12.11 shows such a system. After they are released, the masses are pulled downward by the force of gravity. Notice that the resulting dis- placement of each spring in Fig. 12.11b is measured along local coordinates referenced to its initial position in Fig. 12.11a. As introduced in Chap. 1, Newton’s second law can be employed in conjunction with force balances to develop a mathematical model of the system. For each mass, the second law can be expressed as

m d 2x

d t2 5 FD 2 FU (12.13)

To simplify the analysis, we will assume that all the springs are identical and follow Hooke’s law. A free-body diagram for the fi rst mass is depicted in Fig. 12.12a. The upward force is merely a direct expression of Hooke’s law:

FU 5 k x1 (12.14)

The downward component consists of the two spring forces along with the action of gravity on the mass,

FD 5 k(x2 2 x1) 1 k(x2 2 x1) 5 m1 g (12.15)

Note how the force component of the two springs is proportional to the displacement of the second mass, x2, corrected for the displacement of the fi rst mass, x1. Equations (12.14) and (12.15) can be substituted into Eq. (12.13) to give

m1 d 2x1 d t2

5 2k(x2 2 x1) 1 m1 g 2 k x1 (12.16)

Thus, we have derived a second-order ordinary differential equation to describe the dis- placement of the fi rst mass with respect to time. However, notice that the solution cannot be obtained because the model includes a second dependent variable, x2. Consequently, free-body diagrams must be developed for the second and the third masses (Fig. 12.12b

FIGURE 12.10 The solution for currents and voltages obtained using an elimination method.

V = 153.85 V = 169.23

i = 1.5385

V = 146.15 V = 123.08 V = 0

V = 200

i = 6.1538

cha9792x_ch12_319-340.indd Page 328 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 328 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

12.4 SPRING-MASS SYSTEMS 329

and c) that can be employed to derive

m2 d2x2 dt2

5 k(x3 2 x2) 1 m2g 2 2k(x2 2 x1) (12.17)

and

m3 d2x3 dt2

5 m3 g 2 k(x3 2 x2) (12.18)

m1

m3

m2

m1

m3

0

0

0

x1

x2

x3

kk

k

k

(b)(a)

m2

FIGURE 12.11 A system composed of three masses suspended vertically by a series of springs. (a) The system before release, that is, prior to extension or compression of the springs. (b) The system after release. Note that the positions of the masses are referenced to local coordinates with origins at their position before release.

FIGURE 12.12 Free-body diagrams for the three masses from Fig. 12.11.

m1

k(x2 – x1) m1g k(x2 – x1)

kx1 k(x2 – x1) k(x2 – x1) k(x3 – x2)

m2g k(x3 – x2) m3g

(a) (b) (c)

m2 m3

cha9792x_ch12_319-340.indd Page 329 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 329 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

330 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

Equations (12.16), (12.17), and (12.18) form a system of three differential equations with three unknowns. With the appropriate initial conditions, they could be used to solve for the displacements of the masses as a function of time (that is, their oscillations). We will discuss numerical methods for obtaining such solutions in Part Seven. For the pres- ent, we can obtain the displacements that occur when the system eventually comes to rest, that is, to the steady state. To do this, the derivatives in Eqs. (12.16), (12.17), and (12.18) are set to zero to give

 3k x1 2 2k x2 5 m1 g

22k x1 1 3k x2 2 k x3 5 m2 g

2   k x2 1 k x3 5 m3 g

or, in matrix form,

[K]{X} 5 {W}

where [K], called the stiffness matrix, is

[K ] 5 £ 3k 22k

22k 3k 2k

2k k

§

and {X} and {W} are the column vectors of the unknowns X and the weights mg, respectively.

Solution. At this point, numerical methods can be employed to obtain a solution. If m1 5 2 kg, m2 5 3 kg, m3 5 2.5 kg, and the k’s 5 10 kg/s

2, use LU decomposition to solve for the displacements and generate the inverse of [K]. Substituting the model parameters with g 5 9.81 gives

[K ] 5 £ 30 220

220 30 210

210 10

§  {W} 5 • 19.62

29.43

24.525

LU decomposition can be employed to solve for x1 5 7.36, x2 5 10.06, and x3 5 12.51. These displacements were used to construct Fig. 12.11b. The inverse of the stiffness matrix is computed as

[K ]21 5 £ 0.1 0.1 0.1

0.1 0.15 0.15

0.1 0.15 0.25

§

Each element of this matrix k21ji tells us the displacement of mass i due to a unit force imposed on mass j. Thus, the values of 0.1 in column 1 tell us that a downward unit load to the fi rst mass will displace all of the masses 0.1 m downward. The other elements can be interpreted in a similar fashion. Therefore, the inverse of the stiffness matrix provides a fundamental summary of how the system’s components respond to externally applied forces.

cha9792x_ch12_319-340.indd Page 330 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 330 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 331

PROBLEMS

Chemical/Bio Engineering 12.1 Perform the same computation as in Sec. 12.1, but change c01 to 20 and c03 to 6. Also change the following fl ows: Q01 5 6, Q12 5 4, Q24 5 2, and Q44 5 12. 12.2 If the input to reactor 3 in Sec. 12.1 is decreased 25 percent, use the matrix inverse to compute the percent change in the concen- tration of reactors 2 and 4? 12.3 Because the system shown in Fig. 12.3 is at steady state, what can be said regarding the four fl ows: Q01, Q03, Q44, and Q55? 12.4 Recompute the concentrations for the fi ve reactors shown in Fig. 12.3, if the fl ows are changed to

Q01 5 5  Q31 5 3  Q25 5 2  Q23 5 2

Q15 5 4  Q55 5 3  Q54 5 3  Q34 5 7

Q12 5 4  Q03 5 8  Q24 5 0  Q44 5 10

12.5 Solve the same system as specifi ed in Prob. 12.4, but set Q12 5 Q54 5 0 and Q15 5 Q34 5 3. Assume that the infl ows (Q01, Q03) and outfl ows (Q44, Q55) are the same. Use conservation of fl ow to recompute the values for the other fl ows. 12.6 Figure P12.6 shows three reactors linked by pipes. As indicated, the rate of transfer of chemicals through each pipe is equal to a fl ow rate (Q, with units of cubic meters per second) multiplied by the con- centration of the reactor from which the fl ow originates (c, with units of milligrams per cubic meter). If the system is at a steady state, the transfer into each reactor will balance the transfer out. Develop mass- balance equations for the reactors and solve the three simultaneous linear algebraic equations for their concentrations. 12.7 Employing the same basic approach as in Sec. 12.1, deter- mine the concentration of chloride in each of the Great Lakes using the information shown in Fig. P12.7. 12.8 The Lower Colorado River consists of a series of four reser- voirs as shown in Fig. P12.8. Mass balances can be written for each

reservoir and the following set of simultaneous linear algebraic equations results:

≥ 13.442 0 0 0

213.442 12.252 0 0

0 212.252 12.377 0

0 0 212.377 11.797

¥ μ c1 c2 c3 c4

∂ 5 μ 750.5

300

102

30

where the right-hand-side vector consists of the loadings of chlo- ride to each of the four lakes and c1, c2, c3, and c4 5 the resulting chloride concentrations for Lakes Powell, Mead, Mohave, and Havasu, respectively. (a) Use the matrix inverse to solve for the concentrations in each of

the four lakes. (b) How much must the loading to Lake Powell be reduced in or-

der for the chloride concentration of Lake Havasu to be 75? (c) Using the column-sum norm, compute the condition number

and how many suspect digits would be generated by solving this system.

12.9 A stage extraction process is depicted in Fig. P12.9. In such systems, a stream containing a weight fraction Yin of a chemical enters from the left at a mass fl ow rate of F1. Simultaneously, a solvent carrying a weight fraction Xin of the same chemical enters from the right at a fl ow rate of F2. Thus, for stage i, a mass balance can be represented as

F1Yi21 1 F2 Xi11 5 F1Yi 1 F2 Xi (P12.9.1)

At each stage, an equilibrium is assumed to be established between Yi and Xi as in

K 5 Xi Yi

(P12.9.2)

FIGURE P12.6 Three reactors linked by pipes. The rate of mass transfer through each pipe is equal to the product of fl ow Q and con- centration c of the reactor from which the fl ow originates.

2

3

Q33 = 120 Q13 = 40 Q12 = 80 Q23 = 60 Q21 = 20

Q12c1Q21c2

Q23c2

Q33c3Q13c1 400 mg/s

200 mg/s

1

cha9792x_ch12_319-340.indd Page 331 24/10/13 8:06 PM F-468 cha9792x_ch12_319-340.indd Page 331 24/10/13 8:06 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

332 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

12.10 An irreversible, fi rst-order reaction takes place in four well- mixed reactors (Fig. P12.10),

ASk B

Thus, the rate at which A is transformed to B can be represented as

Rab 5 kVc

The reactors have different volumes, and because they are operated at different temperatures, each has a different reaction rate:

Reactor V, L k, h21

1 25 0.05 2 75 0.1 3 100 0.5

4 25 0.1

Determine the concentration of A and B in each of the reactors at steady state. 12.11 A peristaltic pump delivers a unit fl ow (Q1) of a highly viscous fl uid. The network is depicted in Fig. P12.11. Every pipe section has the same length and diameter. The mass and mechanical energy balance can be simplifi ed to obtain the fl ows in every pipe. Solve the following system of equations to obtain the fl ow in every pipe.

Q3 1 2Q4 2 2Q2 5 0

Q5 1 2Q6 2 2Q4 5 0

3Q7 2 2Q6 5 0

where K is called a distribution coeffi cient. Equation (P12.9.2) can be solved for Xi and substituted into Eq. (P12.9.1) to yield

Yi21 2 a1 1 F2F1 Kb Yi 1 a F2 F1

Kb Yi11 5 0 (P12.9.3) If F1 5 400 kg/h, Yin 5 0.1, F2 5 800 kg/h, Xin 5 0, and K 5 5, determine the values of Yout and Xout if a fi ve-stage reactor is used. Note that Eq. (P12.9.3) must be modifi ed to account for the infl ow weight fractions when applied to the fi rst and last stages.

FIGURE P12.7 A chloride balance for the Great Lakes. Numbered arrows are direct inputs.

QSH = 67 QMH = 36 QHE = 161 QEO = 182 QOO = 212

QSHcS

QMHcM

QHEcH

QEOcE

QOOcO

3850

4720

740

180

710

Superior

Michigan

Huron

SuperiorErie

Ontario

c1

c2

c3

c4

Upper Colorado

River

Lake Mead

Lake Mohave

Lake Havasu

Lake Powell

FIGURE P12.8 The Lower Colorado River.

cha9792x_ch12_319-340.indd Page 332 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 332 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 333

is passed over a liquid fl owing from right to left. The transfer of a chemical from the gas into the liquid occurs at a rate that is propor- tional to the difference between the gas and liquid concentrations in each reactor. At steady state, a mass balance for the fi rst reactor can be written for the gas as

QG cG0 2 QG cG1 1 D(cL1 2 cG1) 5 0

and for the liquid as

QL cL2 2 QL cL1 1 D(cG1 2 cL1) 5 0

where QG and QL are the gas and liquid fl ow rates, respectively, and D 5 the gas-liquid exchange rate. Similar balances can be written for the other reactors. Solve for the concentrations given the follow- ing values: QG 5 2, QL 5 1, D 5 0.8, cG0 5 100, cL6 5 20.

Civil/Environmental Engineering 12.13 A civil engineer involved in construction requires 4800, 5810, and 5690 m3 of sand, fi ne gravel, and coarse gravel, respectively, for

Q1 5 Q2 1 Q3

Q3 5 Q4 1 Q5

Q5 5 Q6 1 Q7

12.12 Figure P12.12 depicts a chemical exchange process consist- ing of a series of reactors in which a gas fl owing from left to right

FIGURE P12.9 A stage extraction process.

Flow = F1

Flow = F2

x2xout x3 xi xi + 1 xn – 1 xn xin

y1yin y2 yi – 1 yi yn – 2 yn – 1 yout

1 02 0n0i n – 1••• •••

1 2 3 4 Qin = 10

Q32 = 5

Q43 = 3

cA,in = 1

FIGURE P12.10

FIGURE P12.11

Q1 Q3 Q5

Q2 Q4 Q6 Q7

cG1cG0 cG2 cG3 cG4 QGQG

QL

cG5

QL

D

cL1 cL2 cL3 cL4 cL5 cL6

FIGURE P12.12

cha9792x_ch12_319-340.indd Page 333 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 333 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

334 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

12.17 In the example for Fig. 12.4, where a 1000-lb downward force is applied at node 1, the external reactions V2 and V3 were calculated. But if the lengths of the truss members had been given, we could have calculated V2 and V3 by utilizing the fact that V2 1 V3 must equal 1000 and by summing moments around node 2. However, because we do know V2 and V3, we can work backward to solve for the lengths of the truss members. Note that because there are three unknown lengths and only two equations, we can solve for only the relationship between lengths. Solve for this relationship. 12.18 Employing the same methods as used to analyze Fig. 12.4, determine the forces and reactions for the truss shown in Fig. P12.18. 12.19 Solve for the forces and reaction for the truss in Fig. P12.19. Determine the matrix inverse for the system. Does the vertical- member force in the middle member seem reasonable? Why?

How many cubic meters must be hauled from each pit in order to meet the engineer’s needs? 12.14 Perform the same computation as in Sec. 12.2, but for the truss depicted in Fig. P12.14. 12.15 Perform the same computation as in Sec. 12.2, but for the truss depicted in Fig. P12.15. 12.16 Calculate the forces and reactions for the truss in Fig. 12.4 if a downward force of 2500 kg and a horizontal force to the right of 2000 kg are applied at node 1.

600

1200

500

30� 45� 45�

FIGURE P12.14

FIGURE P12.19

400 200

45� 60�45� 30�

FIGURE P12.15

a building project. There are three pits from which these materials can be obtained. The composition of these pits is

Sand Fine Gravel Coarse Gravel % % %

Pit 1 52 30 18 Pit 2 20 50 30 Pit 3 25 20 55

FIGURE P12.18

45�

800

250

30�

30�

60� 45� 45�

60�

3500

cha9792x_ch12_319-340.indd Page 334 23/10/13 5:40 PM F-468 cha9792x_ch12_319-340.indd Page 334 23/10/13 5:40 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 335

12.22 A truss is loaded as shown in Fig. P12.22. Using the follow- ing set of equations, solve for the 10 unknowns: AB, BC, AD, BD, CD, DE, CE, Ax, Ay, and Ey.

12.20 As the name implies, indoor air pollution deals with air con- tamination in enclosed spaces such as homes, offi ces, work areas, etc. Suppose that you are designing a ventilation system for a res- taurant as shown in Fig. P12.20. The restaurant serving area con- sists of two square rooms and one elongated room. Room 1 and room 3 have sources of carbon monoxide from smokers and a faulty grill, respectively. Steady-state mass balances can be written for each room. For example, for the smoking section (room 1), the balance can be written as

0 5 Wsmoker 1  Qa ca 2   Qa  c1  1 E13(c3 2 c1)

(load) 1 (infl ow) 2 (outfl ow) 1 (mixing)

or substituting the parameters

225c1 2 25c3 5 2400

Similar balances can be written for the other rooms. (a) Solve for the steady-state concentration of carbon monoxide in

each room. (b) Determine what percent of the carbon monoxide in the kids’

section is due to (i) the smokers, (ii) the grill, and (iii) the air in the intake vents.

(c) If the smoker and grill loads are increased to 2000 and 5000 mg/hr, respectively, use the matrix inverse to determine the in- crease in the concentration in the kids’ section.

(d) How does the concentration in the kids’ area change if a screen is constructed so that the mixing between areas 2 and 4 is de- creased to 5 m3/hr?

12.21 An upward force of 20 kN is applied at the top of a tripod as depicted in Fig. P12.21. Determine the forces in the legs of the tripod.

Qc = 150 m 3/hr

2 (Kids' section)

1 (Smoking section)

Grill load (2000 mg/hr)

Smoker load (1000 mg/hr)

4

25 m3/hr

25 m3/hr

3

Qb = 50 m 3/hr

cb = 2 mg/m 3

Qa = 200 m 3/hr

ca = 2 mg/m 3

Qd = 100 m 3/hr

50 m

3 / h

r

FIGURE P12.20 Overhead view of rooms in a restaurant. The one-way arrows represent volumetric airfl ows, whereas the two-way arrows represent diffusive mixing. The smoker and grill loads add carbon monoxide mass to the system but negligible airfl ow.

D

B

C

A

x

y

0.6 m

2.4 m

0.8 m

0.8 m

1 m

FIGURE P12.21

cha9792x_ch12_319-340.indd Page 335 23/10/13 5:41 PM F-468 cha9792x_ch12_319-340.indd Page 335 23/10/13 5:41 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

336 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

Metal, Plastic, Rubber, Component g/component g/component g/component

1 15 0.30 1.0 2 17 0.40 1.2 3 19 0.55 1.5

If totals of 3.89, 0.095, and 0.282 kg of metal, plastic, and rubber, respectively, are available each day, how many components can be produced per day? 12.27 Determine the currents for the circuit in Fig. P12.27. 12.28 Determine the currents for the circuit in Fig. P12.28. 12.29 The following system of equations was generated by applying the mesh current law to the circuit in Fig. P12.29:

55I1 2 25I4 5 2200

237I3 2 4I4 5 2250

225I1 2 4I3 1 29I4 5 100

Solve for I1, I3, and I4.

Ax 1 AD 5 0

Ay 1 AB 5 0

74 1 BC 1 (3y5)BD 5 0 2AB 2 (4y5)BD 5 0 2BC 1 (3y5)CE 5 0

224 2 CD 2 (4y5)CE 5 0 2AD 1 DE 2 (3y5)BD 5 0 CD 1 (4y5)BD 5 0 2DE 2 (3y5)CE 5 0 Ey 1 (4y5)CE 5 0

Electrical Engineering 12.23 Perform the same computation as in Sec. 12.3, but for the circuit depicted in Fig. P12.23. 12.24 Perform the same computation as in Sec. 12.3, but for the circuit depicted in Fig. P12.24. 12.25 Solve the circuit in Fig. P12.25 for the currents in each wire. Use Gauss elimination with pivoting. 12.26 An electrical engineer supervises the production of three types of electrical components. Three kinds of material—metal, plastic, and rubber—are required for production. The amounts needed to produce each component are

FIGURE P12.22

3 m 3 m

4 m

D A E

C B

54 kN

24 kN

FIGURE P12.23

R = 2 � R = 5 �

R = 20 �3 2 1

4 5 6

R = 5 �

R = 10 � V1 = 200 volts

V6 = 0 volts R = 25 �

FIGURE P12.24

R = 7 �R = 5 � R = 10 �

R = 30 �3 2 1

4 5 6

R = 18 �

R = 35 � V1 = 10 volts

V6 = 200 volts R = 5 �

FIGURE P12.25

20 �

5 �

10 �

10 �

20 �

5 �

5 �

60 �

0 �4 7 9

2

1

8

3 6 15 �

5

V2 = 40V1 = 110

cha9792x_ch12_319-340.indd Page 336 24/10/13 8:05 PM F-468 cha9792x_ch12_319-340.indd Page 336 24/10/13 8:05 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 337

Mechanical/Aerospace Engineering 12.31 Perform the same computation as in Sec. 12.4, but add a third spring between masses 1 and 2 and triple k for all springs. 12.32 Perform the same computation as in Sec. 12.4, but change the masses from 2, 3, and 2.5 kg to 10, 3.5, and 2 kg, respectively. 12.33 Idealized spring-mass systems have numerous applications throughout engineering. Figure P12.33 shows an arrangement of four springs in series being depressed with a force of 2000 kg. At equilibrium, force-balance equations can be developed defi ning the interrelationships between the springs,

k2(x2 2 x1) 5 k1x1 k3(x3 2 x2) 5 k2(x2 2 x1)

k4(x4 2 x3) 5 k3(x3 2 x2)

F 5 k4(x4 2 x3)

where the k’s are spring constants. If k1 through k4 are 150, 50, 75, and 225 N/m, respectively, compute the x’s. 12.34 Three blocks are connected by a weightless cord and rest on an inclined plane (Fig. P12.34a). Employing a procedure similar to the one used in the analysis of the falling parachutists in Example

12.30 The following system of equations was generated by apply- ing the mesh current law to the circuit in Fig. P12.30:

60I1 2 40I2 5 200

240I1 1 150I2 2 100I3 5 0

2100I2 1 130I3 5 230

Solve for I1, I2, and I3.

FIGURE P12.27

15 � 25 � 50 V80 V

5 � 10 � 20 �

+ –

+ –

FIGURE P12.28

20 V

8 �4 �

5 �2 �+–

6 �

i3i1

j2

FIGURE P12.30

200 V

80 V

10 A

20 �

40 �

10 �

100 � 30 �+–

+ –

I1 I2 I3 I4

FIGURE P12.29

100 V

25 �25 �

8 �4 �+–

10 A10 �

20 � I2

I3I4

I1

cha9792x_ch12_319-340.indd Page 337 23/10/13 5:41 PM F-468 cha9792x_ch12_319-340.indd Page 337 23/10/13 5:41 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

338 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

9.11 yields the following set of simultaneous equations (free-body diagrams are shown in Fig. P12.34b):

100a 1 T   5 519.72

50a 2 T 1 R 5 216.55

25a 2 R 5 108.28

Solve for acceleration a and the tensions T and R in the two ropes. 12.35 Perform a computation similar to that called for in Prob. 12.34, but for the system shown in Fig. P12.35. 12.36 Perform the same computation as in Prob. 12.34, but for the system depicted in Fig. P12.36 (angles are 458). 12.37 Consider the three mass-four spring system in Fig. P12.37. Determining the equations of motion from gFx 5 ma, for each mass using its free-body diagram results in the following differential equations:

x$1 1 ak1 1 k2m1 b x1 2 a k2 m1 b x2 5 0

x$2 2 a k2m2b x1 1 a k2 1 k3

m2 b x2 2 a k3m2b x3 5 0

x$3 2 a k3m3b x2 1 a k3 1 k4

m3 b x3 5 0

FIGURE P12.33

F

k4

x4

x

x3

x2

x1

0

k3

k2

k1

FIGURE P12.34

(b)

(a)

10 0 k

g

50 kga,

ac ce

ler at

io n

25 kg

45�

RT

RT

69 2.9

6

692.96

100 � 9.8 = 980

69 2.9

6 � 0.2

5 = 17

3.2 4

34 6.4

8

346.48

50 � 9.8 = 490

34 6.4

8 � 0.3

75 = 1

29 .93

17 3.2

4

173.24

25 � 9.8 = 245

17 3.2

4 �

0.3 75

= 6 4.9

7

cha9792x_ch12_319-340.indd Page 338 23/10/13 5:41 PM F-468 cha9792x_ch12_319-340.indd Page 338 23/10/13 5:41 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 339

where T 5 temperature (8C), x 5 distance along the rod (m), h9 5 a heat transfer coeffi cient between the rod and the ambient air (m22), and Ta 5 the temperature of the surrounding air (8C). This equation can be transformed into a set of linear algebraic equations by using a fi nite divided difference approximation for the second derivative (recall Section 4.1.3),

d 2 T

dx2 5

Ti11 2 2Ti 1 Ti21 ¢x2

where Ti designates the temperature at node i. This approximation can be substituted into Eq. (P12.38.1) to give

2Ti21 1 (2 1 h¿¢x2)Ti 2 Ti11 5 h¿¢x2Ta

This equation can be written for each of the interior nodes of the rod resulting in a tridiagonal system of equations. The fi rst and last nodes at the rod’s ends are fi xed by boundary conditions. (a) Develop an analytical solution for Eq. (P12.38.1) for a

10-m rod with Ta 5 20, T(x 5 0) 5 40, T(x 5 10) 5 200, and h9 5 0.02.

(b) Develop a numerical solution for the same parameter values employed in (a) using a fi nite-difference solution with four in- terior nodes as shown in Fig. P12.38 (Dx 5 2 m).

12.39 The steady-state distribution of temperature on a heated plate can be modeled by the Laplace equation,

0 5 02T 0x2

1 02T 0y2

If the plate is represented by a series of nodes (Fig. P12.39), cen- tered fi nite-divided differences can be substituted for the second derivatives, which results in a system of linear algebraic equations. Use the Gauss-Seidel method to solve for the temperatures of the nodes in Fig. P12.39.

where k1 5 k4 5 10 N/m, k2 5 k3 5 30 N/m, and m1 5 m2 5 m3 5 2 kg. Write the three equations in matrix form:

0 5 [Acceleration vector] 1 [k/m matrix][displacement vector x]

At a specifi c time when x1 5 0.05 m, x2 5 0.04 m, and x3 5 0.03 m, this forms a tridiagonal matrix. Solve for the acceleration of each mass. 12.38 Linear algebraic equations can arise in the solution of differential equations. For example, the following differential equa- tion derives from a heat balance for a long, thin rod (Fig. P12.38):

d 2T

dx2 1 h¿ (Ta 2 T) 5 0 (P12.38.1)

FIGURE P12.35

40 kg

50 kg

10 kg

30� 60�

Friction = 0.5 Friction = 0.3

Friction = 0.2

FIGURE P12.38 A noninsulated uniform rod positioned between two walls of constant but different temperature. The fi nite difference representation employs four interior nodes.

�x T0 = 40 T5 = 200

Ta = 10

Ta = 10

x = 0 x = 10

FIGURE P12.36

Friction = 0.8

Friction = 0.2

8 kg10 kg

15 kg

5 kg

FIGURE P12.37

m1 m2 m3

x1 k2 k3 k4k1

x2 x3

cha9792x_ch12_319-340.indd Page 339 23/10/13 5:41 PM F-468 cha9792x_ch12_319-340.indd Page 339 23/10/13 5:41 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

340 CASE STUDIES: LINEAR ALGEBRAIC EQUATIONS

12.40 A rod on a ball and socket joint is attached to cables A and B, as in Fig. P12.40. (a) If a 50-N force is exerted on the massless rod at G, what is the

tensile force at cables A and B? (b) Solve for the reactant forces at the base of the rod. Call the base

point P.

FIGURE P12.39

T12

T11

T22

T21

200�C

200�C

0�C

0�C

75�C 75�C

25�C 25�C

FIGURE P12.40

Ball and socket

y

x

z

50 N 2 m

2 m

2 m

1 m B

2 m 1 m

A

cha9792x_ch12_319-340.indd Page 340 23/10/13 5:41 PM F-468 cha9792x_ch12_319-340.indd Page 340 23/10/13 5:41 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.1 CURRENT 1ST LEVEL HEAD 341

341

PT3.4 TRADE-OFFS

Table PT3.2 provides a summary of the trade-offs involved in solving simultaneous linear algebraic equations. Two methods—graphical and Cramer’s rule—are limited to small (# 3) numbers of equations and thus have little utility for practical problem solv- ing. However, these techniques are useful didactic tools for understanding the behavior of linear systems in general. The numerical methods themselves are divided into two general categories: exact and approximate methods. As the name implies, the former are intended to yield exact answers. However, because they are affected by round-off errors, they sometimes yield imprecise results. The magnitude of the round-off error varies from system to system and is dependent on a number of factors. These include the system’s dimensions, its condition, and whether the matrix of coeffi cients is sparse or full. In addition, computer precision will affect round-off error. It is recommended that a pivoting strategy be employed in any computer program implementing exact elimination methods. The inclusion of such a strategy minimizes round-off error and avoids problems such as division by zero. All other things being equal, LU decomposition–based algorithms are the methods of choice because of their effi ciency and fl exibility.

TABLE PT3.2 Comparison of the characteristics of alternative methods for fi nding solutions of simultaneous linear algebraic equations.

Breadth of Programming Method Stability Precision Application Effort Comments

Graphical — Poor Limited — May take more time than the numerical method, but can be

useful for visualization Cramer’s rule — Affected by Limited — Excessive computational effort round-off error required for more than three

equations Gauss elimination (with — Affected by General Moderate partial pivoting) round-off error LU decomposition — Affected by General Moderate Preferred elimination method; allows round-off error computation of matrix inverse Gauss-Seidel May not Excellent Appropriate only Easy converge if not for diagonally diagonally dominant dominant systems

EPILOGUE: PART THREE

cha9792x_ep03_341-343.indd Page 341 23/10/13 5:43 PM F-468 cha9792x_ep03_341-343.indd Page 341 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

342 EPILOGUE: PART THREE

Although elimination methods have great utility, their use of the entire matrix of coeffi cients can be somewhat limiting when dealing with very large, sparse systems. This is due to the fact that large portions of computer memory would be devoted to storage of meaningless zeros. For banded systems, techniques are available to implement elimination methods without having to store the entire coeffi cient matrix. The approximate technique described in this book is called the Gauss-Seidel method. It differs from the exact techniques in that it employs an iterative scheme to obtain progressively closer estimates of the solution. Thus, the effect of round-off is a moot point with the Gauss-Seidel method because the iterations can be continued as long as is necessary to obtain the desired precision. In addition, versions of the Gauss- Seidel method can be developed to effi ciently utilize computer storage requirements for sparse systems. Consequently, the Gauss-Seidel technique has utility for large sys- tems of equations where storage requirements would pose signifi cant problems for the exact techniques. The disadvantage of the Gauss-Seidel method is that it does not always converge or sometimes converges slowly on the true solution. It is strictly reliable only for those systems that are diagonally dominant. However, relaxation methods are available that sometimes offset these disadvantages. In addition, because many sets of linear algebraic equations originating from physical systems exhibit diagonal dominance, the Gauss- Seidel method has great utility for engineering problem solving. In summary, a variety of factors will bear on your choice of a technique for a par- ticular problem involving linear algebraic equations. However, as outlined above, the size and sparseness of the system are particularly important factors in determining your choice.

PT3.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Every part of this book includes a section that summarizes important formulas. Although Part Three does not really deal with single formulas, we have used Table PT3.3 to sum- marize the algorithms that were covered. The table provides an overview that should be helpful for review and in elucidating the major differences between the methods.

PT3.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

General references on the solution of simultaneous linear equations can be found in Fadeev and Fadeeva (1963), Stewart (1973), Varga (1962), and Young (1971). Ralston and Rabinowitz (1978) provide a general summary. Many advanced techniques are available to increase the savings in time and/or space when solving linear algebraic equations. Most of these focus on exploiting properties of the equations such as symmetry and bandedness. In particular, algorithms are available to operate on sparse matrices to convert them to a minimum banded format. Jacobs (1977) and Tewarson (1973) include information on this area. Once they are in a mini- mum banded format, there are a variety of effi cient solution strategies that are employed such as the active column storage approach of Bathe and Wilson (1976). Aside from n 3 n sets of equations, there are other systems where the number of equations, m, and number of unknowns, n, are not equal. Systems where m , n are called underdetermined. In such cases, there can be either no solution or else more than

cha9792x_ep03_341-343.indd Page 342 23/10/13 5:43 PM F-468 cha9792x_ep03_341-343.indd Page 342 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT3.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 343

one. Systems where m . n are called overdetermined. For such situations, there is in general no exact solution. However, it is often possible to develop a compromise solution that attempts to determine answers that come “closest” to satisfying all the equations simultaneously. A common approach is to solve the equation in a “least-squares” sense (Lawson and Hanson, 1974; Wilkinson and Reinsch, 1971). Alternatively, linear program- ming methods can be used where the equations are solved in an “optimal” sense by minimizing some objective function (Dantzig, 1963; Luenberger, 1984; and Rabinowitz, 1968). We describe this approach in detail in Chap. 15.

TABLE PT3.3 Summary of important information presented in Part Three.

Potential Problems and Method Procedure Remedies

Gauss elimination

LU decomposition

Gauss-Seidel method

Problems: III conditioning Round-off Division by zero Remedies: Higher precision Partial pivoting

Problems: III conditioning Round-off Division by zero Remedies: Higher precision Partial pivoting

Problems: Divergent or converges slowly Remedies: Diagonal dominance Relaxation

£ a11 a12 a13 0 c1 a21 a22 a23 0 c2 a31 a32 a33 0 c3

§ 1 £ a11 a12 a13 Z c1

a'22 a'23 Z c'2 a''33 Z c''3

§ 1 x3 5 c''3ya''33 x2 5 1c'2 2 a'23x32ya'22 x1 5 1c1 2 a12x1 2 a13x32ya11

Decomposition Back Substitution

£ a11 a12 a13 a21 a22 a23 a31 a32 a33

§ 1 £ 1 0 0 l21 1 0 l31 l32 1

§ • d1 d2 d3 ¶ 5 •

c1 c2 c3 ¶ 1 £

u11 u12 u13 0 u22 u23 0 0 u33

§ • x1 x2 x3 ¶ 5 •

d1 d2 d3 ¶ 5 •

x1 x2 x3 ¶

Forward Substitution

x i1 5 (c1 2 a12x i212 2 a13x i213 )ya11 x i2 5 (c2 2 a21x i1 2 a23x i213 )ya22 x i3 5 (c3 2 a31x i1 2 a32x i2)ya33

¶ ` x i i 2 xi21i

xii ` 100% , es

for all x'i s

continue iteratively until

cha9792x_ep03_341-343.indd Page 343 23/10/13 5:43 PM F-468 cha9792x_ep03_341-343.indd Page 343 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART FOUR

cha9792x_p04_344-354.indd Page 344 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 344 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

345

PT4.1 MOTIVATION

Root location (Part 2) and optimization are related in the sense that both involve guessing and searching for a point on a function. The fundamental difference between the two types of problems is illustrated in Fig. PT4.1. Root location involves searching for zeros of a function or functions. In contrast, optimization involves searching for either the minimum or the maximum. The optimum is the point where the curve is fl at. In mathematical terms, this corre- sponds to the x value where the derivative f9(x) is equal to zero. Additionally, the second derivative, f 0(x), indicates whether the optimum is a minimum or a maximum: if f 0(x) , 0, the point is a maximum; if f 0(x) . 0, the point is a minimum. Now, understanding the relationship between roots and optima would suggest a pos- sible strategy for fi nding the latter. That is, you can differentiate the function and locate the root (that is, the zero) of the new function. In fact, some optimization methods seek to fi nd an optima by solving the root problem: f9(x) 5 0. It should be noted that such searches are often complicated because f9(x) is not available analytically. Thus, one must sometimes use fi nite-difference approximations to estimate the derivative. Beyond viewing optimization as a roots problem, it should be noted that the task of locating optima is aided by some extra mathematical structure that is not part of simple root fi nding. This tends to make optimization a more tractable task, particularly for multidimensional cases.

OPTIMIZATION

FIGURE PT4.1 A function of a single variable illustrating the difference between roots and optima.

Maximum

Minimum

0 Root

Root

Root

f (x)

x

f �(x) = 0 f �(x) � 0

f �(x) = 0 f �(x) � 0

f (x) = 0

cha9792x_p04_344-354.indd Page 345 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 345 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

346 OPTIMIZATION

PT4.1.1 Noncomputer Methods and History

As mentioned above, differential calculus methods are still used to determine optimum solu- tions. All engineering and science students recall working maxima-minima problems by determining fi rst derivatives of functions in their calculus courses. Bernoulli, Euler, Lagrange, and others laid the foundations of the calculus of variations, which deals with the minimiza- tion of functions. The Lagrange multiplier method was developed to optimize constrained problems, that is, optimization problems where the variables are bounded in some way. The fi rst major advances in numerical approaches occurred only with the develop- ment of digital computers after World War II. Koopmans in the United Kingdom and Kantorovich in the former Soviet Union independently worked on the general problem of least-cost distribution of supplies and products. In 1947, Koopman’s student Dantzig invented the simplex procedure for solving linear programming problems. This approach paved the way for other methods of constrained optimization by a number of investiga- tors, notably Charnes and his coworkers. Approaches for unconstrained optimization also developed rapidly following the widespread availability of computers.

PT4.1.2 Optimization and Engineering Practice

Most of the mathematical models we have dealt with to this point have been descriptive models. That is, they have been derived to simulate the behavior of an engineering device or system. In contrast, optimization typically deals with fi nding the “best result,” or opti- mum solution, of a problem. Thus, in the context of modeling, they are often termed prescriptive models since they can be used to prescribe a course of action or the best design. Engineers must continuously design devices and products that perform tasks in an effi cient fashion. In doing so, they are constrained by the limitations of the physical world. Further, they must keep costs down. Thus, they are always confronting optimiza- tion problems that balance performance and limitations. Some common instances are listed in Table PT4.1. The following example has been developed to help you get a feel for the way in which such problems might be formulated.

TABLE PT4.1 Some common examples of optimization problems in engineering.

• Design aircraft for minimum weight and maximum strength. • Optimal trajectories of space vehicles. • Design civil engineering structures for minimum cost. • Design water-resource projects like dams to mitigate fl ood damage while yielding maximum hydropower. • Predict structural behavior by minimizing potential energy. • Material-cutting strategy for minimum cost. • Design pump and heat transfer equipment for maximum effi ciency. • Maximize power output of electrical networks and machinery while minimizing heat generation. • Shortest route of salesperson visiting various cities during one sales trip. • Optimal planning and scheduling. • Statistical analysis and models with minimum error. • Optimal pipeline networks. • Inventory control. • Maintenance planning to minimize cost. • Minimize waiting and idling times. • Design waste treatment systems to meet water-quality standards at least cost.

cha9792x_p04_344-354.indd Page 346 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 346 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT4.1 MOTIVATION 347

EXAMPLE PT4.1 Optimization of Parachute Cost

Problem Statement. Throughout the rest of the book, we have used the falling para- chutist to illustrate the basic problem areas of numerical methods. You may have noticed that none of these examples concentrate on what happens after the chute opens. In this example, we will examine a case where the chute has opened and we are interested in predicting impact velocity at the ground. You are an engineer working for an agency planning to airlift supplies to refugees in a war zone. The supplies will be dropped at low altitude (500 m) so that the drop is not detected and the supplies fall as close as possible to the refugee camp. The chutes open immediately upon leaving the plane. To reduce damage, the vertical velocity on impact must be below a critical value of yc 5 20 m/s. The parachute used for the drop is depicted in Fig. PT4.2. The cross-sectional area of the chute is that of a half sphere,

A 5 2pr2 (PT4.1)

The length of each of the 16 cords connecting the chute to the mass is related to the chute radius by

/ 5 12r (PT4.2)

You know that the drag force for the chute is a linear function of its cross-sectional area described by the following formula

c 5 kc A (PT4.3)

where c 5 drag coeffi cient (kg/s) and kc 5 a proportionality constant parameterizing the effect of area on drag [kg/(s ? m2)]. Also, you can divide the payload into as many parcels as you like. That is, the mass of each individual parcel can be calculated as

m 5 Mt n

FIGURE PT4.2 A deployed parachute.

m

r

cha9792x_p04_344-354.indd Page 347 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 347 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

348 OPTIMIZATION

where m 5 mass of an individual parcel (kg), Mt 5 total load being dropped (kg), and n 5 total number of parcels. Finally, the cost of each chute is related to chute size in a nonlinear fashion,

Cost per chute 5 c0 1 c1/ 1 c2A2 (PT4.4)

where c0, c1, and c2 5 cost coeffi cients. The constant term, c0, is the base price for the chutes. The nonlinear relationship between cost and area exists because larger chutes are much more diffi cult to construct than small chutes. Determine the size (r) and number of chutes (n) that result in minimum cost while at the same time meeting the requirement of having a suffi ciently small impact velocity.

Solution. The objective here is to determine the number and size of parachutes to minimize the cost of the airlift. The problem is constrained because the parcels must have an impact velocity less than a critical value. The cost can be computed by multiplying the cost of the individual parachute [Eq. (PT4.4)] by the number of parachutes (n). Thus, the function you wish to minimize, which is formally called the objective function, is written as

Minimize C 5 n(c0 1 c1/ 1 c2A2) (PT4.5)

where C 5 cost ($) and A and / are calculated by Eqs. (PT4.1) and (PT4.2), respectively. Next, we must specify the constraints. For this problem there are two constraints. First, the impact velocity must be equal to or less than the critical velocity,

y # yc (PT4.6)

Second, the number of parcels must be an integer and greater than or equal to 1,

n $ 1 (PT4.7)

where n is an integer. At this point, the optimization problem has been formulated. As can be seen, it is a nonlinear constrained problem. Although the problem has been broadly formulated, one more issue must be addressed: How do we determine the impact velocity y? Recall from Chap. 1 that the velocity of a falling object can be computed with

y 5 gm

c (1 2 e2 (cym)t) (1.10)

where y 5 velocity (m/s), g 5 acceleration of gravity (m/s2), m 5 mass (kg), and t 5 time (s). Although Eq. (1.10) provides a relationship between y and t, we need to know how long the mass falls. Therefore, we need a relationship between the drop distance z and the time of fall t. The drop distance can be calculated from the velocity in Eq. (1.10) by integration

z 5 # t

0

gm

c (1 2 e2(c/m)t) dt (PT4.8)

This integral can be evaluated to yield

z 5 z0 2 gm

c t 1

gm2

c2 (1 2 e2(c/m)t) (PT4.9)

cha9792x_p04_344-354.indd Page 348 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 348 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT4.1 MOTIVATION 349

where z0 5 initial height (m). This function, as plotted in Fig. PT4.3, provides a way to predict z given knowledge of t. However, we do not need z as a function of t to solve this problem. Rather, we need to compute the time required for the parcel to fall the distance z0. Thus, we recognize that we must reformulate Eq. (PT4.9) as a root-fi nding problem. That is, we must solve for the time at which z goes to zero,

f(t) 5 0 5 z0 2 gm

c t 1

gm2

c2 (1 2 e2(cym)t) (PT4.10)

Once the time to impact is computed, we can substitute it into Eq. (1.10) to solve for the impact velocity. The fi nal specifi cation of the problem, therefore, would be

Minimize C 5 n(c0 1 c1/ 1 c2 A2) (PT4.11)

subject to

y # yc (PT4.12)

n $ 1 (PT4.13)

where

A 5 2pr2 (PT4.14)

/ 5 12r (PT4.15) c 5 kc A (PT4.16)

m 5 Mt n

(PT4.17)

FIGURE PT4.3 The height z and velocity v of a deployed parachute as it falls to earth (z 5 0).

5 10 t (s)

v (m/s)

z (m)

15

Impact

0 0

200

400

600

cha9792x_p04_344-354.indd Page 349 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 349 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

350 OPTIMIZATION

t 5 root c z0 2 gmc t 1 gm2

c2 (1 2 e2(cym)t) d (PT4.18)

y 5 gm

c (1 2 e2(cym)t) (PT4.19)

We will solve this problem in Example 15.4 in Chap. 15. For the time being recog- nize that it has most of the fundamental elements of other optimization problems you will routinely confront in engineering practice. These are

• The problem will involve an objective function that embodies your goal. • There will be a number of design variables. These variables can be real numbers or

they can be integers. In our example, these are r (real) and n (integer). • The problem will include constraints that reflect the limitations you are working under.

We should make one more point before proceeding. Although the objective function and constraints may superfi cially appear to be simple equations [e.g., Eq. (PT4.12)], they may in fact be the “tip of the iceberg.” That is, they may be underlain by complex de- pendencies and models. For instance, as in our example, they may involve other numeri- cal methods [Eq. (PT4.18)]. This means that the functional relationships you will be using could actually represent large and complicated calculations. Thus, techniques that can fi nd the optimal solution, while minimizing function evaluations, can be extremely valuable.

PT4.2 MATHEMATICAL BACKGROUND

There are a number of mathematical concepts and operations that underlie optimization. Because we believe that they will be more relevant to you in context, we will defer discussion of specifi c mathematical prerequisites until they are needed. For example, we will discuss the important concepts of the gradient and Hessians at the beginning of Chap. 14 on multivariate unconstrained optimization. In the meantime, we will limit ourselves here to the more general topic of how optimization problems are classifi ed. An optimization or mathematical programming problem generally can be stated as:

Find x, which minimizes or maximizes f(x)

subject to

di (x) # ai  i 5 1, 2, p , m (PT4.20) ei (x) 5 bi  i 5 1, 2, p , p (PT4.21)

where x is an n-dimensional design vector, f (x) is the objective function, di(x) are inequal- ity constraints, ei(x) are equality constraints, and ai and bi are constants. Optimization problems can be classifi ed on the basis of the form of f(x):

• If f(x) and the constraints are linear, we have linear programming. • If f(x) is quadratic and the constraints are linear, we have quadratic programming. • If f(x) is not linear or quadratic and/or the constraints are nonlinear, we have nonlinear

programming.

cha9792x_p04_344-354.indd Page 350 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 350 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT4.3 ORIENTATION 351

Further, when Eqs. (PT4.20) and (PT4.21) are included, we have a constrained optimiza- tion problem; otherwise, it is an unconstrained optimization problem. Note that for constrained problems, the degrees of freedom are given by n2p2m. Generally, to obtain a solution, p 1 m must be # n. If p 1 m . n, the problem is said to be overconstrained. Another way in which optimization problems are classifi ed is by dimensionality. This is most commonly done by dividing them into one-dimensional and multidimen- sional problems. As the name implies, one-dimensional problems involve functions that depend on a single dependent variable. As in Fig. PT4.4a, the search then consists of climbing or descending one-dimensional peaks and valleys. Multidimensional problems involve functions that depend on two or more dependent variables. In the same spirit, a two-dimensional optimization can again be visualized as searching out peaks and valleys (Fig. PT4.4b). However, just as in real hiking, we are not constrained to walk a single direction, instead the topography is examined to effi ciently reach the goal. Finally, the process of fi nding a maximum versus fi nding a minimum is essentially identical because the same value, x*, both minimizes f(x) and maximizes 2f(x). This equivalence is illustrated graphically for a one-dimensional function in Fig. PT4.4a.

PT4.3 ORIENTATION

Some orientation is helpful before proceeding to the numerical methods for optimization. The following is intended to provide an overview of the material in Part Four. In addi- tion, some objectives have been included to help you focus your efforts when studying the material.

FIGURE PT4.4 (a) One-dimensional optimization. This fi gure also illustrates how minimization of f(x) is equivalent to the maximization of 2f(x). (b) Two-dimensional optimization. Note that this fi gure can be taken to represent either a maximization (contours increase in elevation up to the maximum like a mountain) or a minimization (contours decrease in elevation down to the minimum like a valley).

x*

x*

x

x

(b)(a)

Optimum f (x*, y*)

Minimum f (x)

f (x)

– f (x)

Maximum – f (x)

f (x, y)

f (x)

y*

y

cha9792x_p04_344-354.indd Page 351 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 351 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

352 OPTIMIZATION

PT4.3.1 Scope and Preview

Figure PT4.5 is a schematic representation of the organization of Part Four. Examine this fi gure carefully, starting at the top and working clockwise. After the present introduction, Chap. 13 is devoted to one-dimensional unconstrained optimization. Methods are presented to fi nd the minimum or maximum of a function of a single variable. Three methods are covered: golden-section search, parabolic interpola- tion, and Newton’s method. An advanced hybrid approach, Brent’s method, that combines the reliability of the golden-section search with the speed of parabolic interpolation is also described. Chapter 14 covers two general types of methods to solve multidimensional uncon- strained optimization problems. Direct methods such as random searches, univariate searches, and pattern searches do not require the evaluation of the function’s derivatives. On the other hand, gradient methods use either fi rst and sometimes second derivatives to fi nd the optimum. The chapter introduces the gradient and the Hessian, which are multidimensional representations of the fi rst and second derivatives. The method of steep- est ascent/descent is then covered in some detail. This is followed by descriptions of some advanced methods: conjugate gradient, Newton’s method, Marquardt’s method, and quasi-Newton methods. Chapter 15 is devoted to constrained optimization. Linear programming is described in detail using both a graphical representation and the simplex method. The detailed analysis of nonlinear constrained optimization is beyond this book’s scope, but we pro- vide an overview of the major approaches. In addition, we illustrate how such problems (along with the problems covered in Chaps. 13 and 14) can be obtained with software packages such as Excel, MATLAB, and Mathcad. Chapter 16 extends the above concepts to actual engineering problems. Engineering applications are used to illustrate how optimization problems are formulated and provide insight into the application of the solution techniques in professional practice. An epilogue is included at the end of Part Four. It contains an overview of the methods discussed in Chaps. 13, 14, and 15. This overview includes a description of trade-offs related to the proper use of each technique. This section also provides refer- ences for some numerical methods that are beyond the scope of this text.

PT4.3.2 Goals and Objectives

Study Objectives. After completing Part Four, you should have suffi cient information to successfully approach a wide variety of engineering problems dealing with optimiza- tion. In general, you should have mastered the techniques, have learned to assess their reliability, and be capable of analyzing alternative methods for any particular problem. In addition to these general goals, the specifi c concepts in Table PT4.2 should be as- similated for a comprehensive understanding of the material in Part Four.

Computer Objectives. You should be able to write a subprogram to implement a simple one-dimensional (like golden-section search or parabolic interpolation) and multidimen- sional (like the random-search method) search. In addition, software packages such as Excel, MATLAB, or Mathcad have varying capabilities for optimization. You can use this part of the book to become familiar with these capabilities.

cha9792x_p04_344-354.indd Page 352 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 352 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT4.3 ORIENTATION 353

FIGURE PT4.5 Schematic of the organization of the material in Part Four: Optimization.

CHAPTER 13 One-Dimensional

Unconstrained Optimization

PART 4 Optimization

CHAPTER 14 Multidimensional

Unconstrained Optimization

CHAPTER 15 Constrained Optimization

CHAPTER 16 Case Studies

EPILOGUE

14.2 Gradient methods

14.1 Direct

methods

PT 4.2 Mathematical background

PT 4.5 Additional references

16.4 Mechanical engineering

16.3 Electrical

engineering

16.2 Civil

engineering 16.1

Chemical engineering

15.1 Linear

programming 15.3

Software packages 15.2

Nonlinear constrained

PT 4.4 Trade-offs

PT 4.3 Orientation

PT 4.1 Motivation

13.2 Parabolic

interpolation

13.3 Newton's method

13.4 Brent's method

13.1 Golden-section

search

cha9792x_p04_344-354.indd Page 353 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 353 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

354 OPTIMIZATION

TABLE PT4.2 Specifi c study objectives for Part Four.

1. Understand why and where optimization occurs in engineering problem solving. 2. Understand the major elements of the general optimization problem: objective function, decision

variables, and constraints. 3. Be able to distinguish between linear and nonlinear optimization, and between constrained and

unconstrained problems. 4. Be able to defi ne the golden ratio and understand how it makes one-dimensional optimization

effi cient. 5. Locate the optimum of a single variable function with the golden-section search, parabolic

interpolation, and Newton’s method. Also, recognize the trade-offs among these approaches, with particular attention to initial guesses and convergence.

6. Understand how Brent’s optimization method combines the reliability of the golden-section search with the speed of parabolic interpolation.

7. Be capable of writing a program and solving for the optimum of a multivariable function using random searching.

8. Understand the ideas behind pattern searches, conjugate directions, and Powell’s method. 9. Be able to defi ne and evaluate the gradient and Hessian of a multivariable function both

analytically and numerically. 10. Compute by hand the optimum of a two-variable function using the method of steepest ascent/

descent. 11. Understand the basic ideas behind the conjugate gradient, Newton’s, Marquardt’s, and quasi-

Newton methods. In particular, understand the trade-offs among the approaches and recognize how each improves on the steepest ascent/descent.

12. Be capable of recognizing and setting up a linear programming problem to represent applicable engineering problems.

13. Be able to solve a two-dimensional linear programming problem with both the graphical and simplex methods.

14. Understand the four possible outcomes of a linear programming problem. 15. Be able to set up and solve nonlinear constrained optimization problems using a software package.

cha9792x_p04_344-354.indd Page 354 23/10/13 5:43 PM F-468 cha9792x_p04_344-354.indd Page 354 23/10/13 5:43 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13

355

C H A P T E R 13

One-Dimensional Unconstrained Optimization

This section will describe techniques to fi nd the minimum or maximum of a function of a single variable, f(x). A useful image in this regard is the one-dimensional, “roller coaster”– like function depicted in Fig. 13.1. Recall from Part Two that root location was complicated by the fact that several roots can occur for a single function. Similarly, both local and global optima can occur in optimization. Such cases are called multimodal. In almost all instances, we will be interested in fi nding the absolute highest or lowest value of a func- tion. Thus, we must take care that we do not mistake a local result for the global optimum. Distinguishing a global from a local extremum can be a very diffi cult problem for the general case. There are three usual ways to approach this problem. First, insight into the behavior of low-dimensional functions can sometimes be obtained graphically. Sec- ond, fi nding optima based on widely varying and perhaps randomly generated starting guesses, and then selecting the largest of these as global. Finally, perturbing the starting point associated with a local optimum and seeing if the routine returns a better point or always returns to the same point. Although all these approaches can have utility, the fact is that in some problems (usually the large ones), there may be no practical way to ensure that you have located a global optimum. However, although you should always

FIGURE 13.1 A function that asymptotically approaches zero at plus and minus q and has two maximum and two minimum points in the vicinity of the origin. The two points to the right are local optima, whereas the two to the left are global.

Local maximum

Local minimum

Global minimum

Global maximum

f (x)

x

cha9792x_ch13_355-369.indd Page 355 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 355 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

356 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

be sensitive to the issue, it is fortunate that there are numerous engineering problems where you can locate the global optimum in an unambiguous fashion. Just as in root location, optimization in one dimension can be divided into bracket- ing and open methods. As described in the next section, the golden-section search is an example of a bracketing method that depends on initial guesses that bracket a single optimum. This is followed by an alternative approach, parabolic interpolation, which often converges faster than the golden-section search, but sometimes diverges. Another method described in this chapter is an open method based on the idea from calculus that the minimum or maximum can be found by solving f9(x) 5 0. This reduces the optimization problem to fi nding the root of f9(x) using techniques of the sort described in Part Two. We will demonstrate one version of this approach—Newton’s method. Finally, an advanced hybrid approach, Brent’s method, is described. This ap- proach combines the reliability of the golden-section search with the speed of para- bolic interpolation.

13.1 GOLDEN-SECTION SEARCH

In solving for the root of a single nonlinear equation, the goal was to fi nd the value of the variable x that yields a zero of the function f(x). Single-variable optimization has the goal of fi nding the value of x that yields an extremum, either a maximum or minimum of f(x). The golden-section search is a simple, general-purpose, single-variable search tech- nique. It is similar in spirit to the bisection approach for locating roots in Chap. 5. Recall that bisection hinged on defi ning an interval, specifi ed by a lower guess (xl) and an upper guess (xu), that bracketed a single root. The presence of a root between these bounds was verifi ed by determining that f(xl) and f(xu) had different signs. The root was then estimated as the midpoint of this interval,

xr 5 xl 1 xu

2

The fi nal step in a bisection iteration involved determining a new smaller bracket. This was done by replacing whichever of the bounds xl or xu had a function value with the same sign as f(xr). One advantage of this approach was that the new value xr replaced one of the old bounds. Now we can develop a similar approach for locating the optimum of a one-dimensional function. For simplicity, we will focus on the problem of fi nding a maximum. When we discuss the computer algorithm, we will describe the minor modifi cations needed to simu- late a minimum. As with bisection, we can start by defi ning an interval that contains a single answer. That is, the interval should contain a single maximum, and hence is called unimodal. We can adopt the same nomenclature as for bisection, where xl and xu defi ned the lower and upper bounds, respectively, of such an interval. However, in contrast to bisection, we need a new strategy for fi nding a maximum within the interval. Rather than using only two function values (which are suffi cient to detect a sign change, and hence a zero), we would need three function values to detect whether a maximum occurred. Thus, an ad- ditional point within the interval has to be chosen. Next, we have to pick a fourth point.

cha9792x_ch13_355-369.indd Page 356 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 356 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.1 GOLDEN-SECTION SEARCH 357

Then the test for the maximum could be applied to discern whether the maximum occurred within the fi rst three or the last three points. The key to making this approach effi cient is the wise choice of the intermediate points. As in bisection, the goal is to minimize function evaluations by replacing old values with new values. This goal can be achieved by specifying that the following two conditions hold (Fig. 13.2):

/0 5 /1 1 /2 (13.1)

/1 /0

5 /2 /1

(13.2)

The fi rst condition specifi es that the sum of the two sublengths /1 and /2 must equal the original interval length. The second says that the ratio of the lengths must be equal. Equation (13.1) can be substituted into Eq. (13.2),

/1 /1 1 /2

5 /2 /1

(13.3)

If the reciprocal is taken and R 5 /2 y  /1, we arrive at

1 1 R 5 1

R (13.4)

or

R2 1 R 2 1 5 0 (13.5)

which can be solved for the positive root

R 5 21 1 11 2 4(21)

2 5 15 2 1

2 5 0.61803p (13.6)

FIGURE 13.2 The initial step of the golden-section search algorithm involves choosing two interior points according to the golden ratio.

Maximum

First iteration

Second iteration

f (x)

xxuxl

�0 �1

�2

�2

cha9792x_ch13_355-369.indd Page 357 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 357 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

358 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

This value, which has been known since antiquity, is called the golden ratio (see Box 13.1). Because it allows optima to be found effi ciently, it is the key element of the golden-section method we have been developing conceptually. Now let us derive an al- gorithm to implement this approach on the computer. As mentioned above and as depicted in Fig. 13.4, the method starts with two initial guesses, xl and xu, that bracket one local extremum of f(x). Next, two interior points x1 and x2 are chosen according to the golden ratio,

d 5 15 2 1

2 (xu 2 xl)

x1 5 xl 1 d

x2 5 xu 2 d

The function is evaluated at these two interior points. Two results can occur:

1. If, as is the case in Fig. 13.4, f(x1) . f(x2), then the domain of x to the left of x2, from xl to x2, can be eliminated because it does not contain the maximum. For this case, x2 becomes the new xl for the next round.

2. If f(x2) . f(x1), then the domain of x to the right of x1, from x1 to xu would have been eliminated. In this case, x1 becomes the new xu for the next round.

Box 13.1 The Golden Ratio and Fibonacci Numbers

In many cultures, certain numbers are ascribed qualities. For example, we in the West are all familiar with “Lucky 7” and “Friday the 13th.” Ancient Greeks called the following number the “golden ratio:”

15 2 1

2 5 0.61803 p

This ratio was employed for a number of purposes, including the development of the rectangle in Fig. 13.3. These proportions were considered aesthetically pleasing by the Greeks. Among other things, many of their temples followed this shape. The golden ratio is related to an important mathematical series known as the Fibonacci numbers, which are

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, p

Thus, each number after the fi rst two represents the sum of the preceding two. This sequence pops up in many diverse areas of sci- ence and engineering. In the context of the present discussion, an interesting property of the Fibonacci sequence relates to the ratio of consecutive numbers in the sequence; that is, 0y1 5 0, 1y1 5 1, 1y2 5 0.5, 2y3 5 0.667, 3y5 5 0.6, 5y8 5 0.625, 8y13 5 0.615, and so on. As one proceeds, the ratio of consecutive numbers ap- proaches the golden ratio!

FIGURE 13.3 The Parthenon in Athens, Greece, was constructed in the 5th century B.C. Its front dimensions can be fi t almost exactly within a golden rectangle.

0.61803

1

cha9792x_ch13_355-369.indd Page 358 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 358 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.1 GOLDEN-SECTION SEARCH 359

Now, here is the real benefi t from the use of the golden ratio. Because the original x1 and x2 were chosen using the golden ratio, we do not have to recalculate all the func- tion values for the next iteration. For example, for the case illustrated in Fig. 13.4, the old x1 becomes the new x2. This means that we already have the value for the new f(x2), since it is the same as the function value at the old x1. To complete the algorithm, we now only need to determine the new x1. This is done with the same proportionality as before,

x1 5 xl 1 15 2 1

2 (xu 2 xl)

A similar approach would be used for the alternate case where the optimum fell in the left subinterval. As the iterations are repeated, the interval containing the extremum is reduced rap- idly. In fact, each round the interval is reduced by a factor of the golden ratio (about 61.8%). That means that after 10 rounds, the interval is shrunk to about 0.61810 or 0.008 or 0.8% of its initial length. After 20 rounds, it is about 0.0066%. This is not quite as good as the reduction achieved with bisection, but this is a harder problem.

FIGURE 13.4 (a) The initial step of the golden-section search algorithm involves choosing two interior points ac- cording to the golden ratio. (b) The second step involves defi ning a new interval that includes the optimum.

Extremum (maximum)Eliminate

f (x)

xx1xl d

xux2 d

(a)

f (x)

xx2 x1xl

Old x1Old x2

xu

(b)

cha9792x_ch13_355-369.indd Page 359 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 359 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

360 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

EXAMPLE 13.1 Golden-Section Search

Problem Statement. Use the golden-section search to fi nd the maximum of

f(x) 5 2 sin x 2 x2

10

within the interval xl 5 0 and xu 5 4.

Solution. First, the golden ratio is used to create the two interior points

d 5 15 2 1

2 (4 2 0) 5 2.472

x1 5 0 1 2.472 5 2.472

x2 5 4 2 2.472 5 1.528

The function can be evaluated at the interior points

f(x2) 5 f(1.528) 5 2 sin(1.528) 2 1.5282

10 5 1.765

f(x1) 5 f(2.472) 5 0.63

Because f(x2) . f(x1), the maximum is in the interval defi ned by xl, x2, and x1. Thus, for the new interval, the lower bound remains xl 5 0, and x1 becomes the upper bound, that is, xu 5 2.472. In addition, the former x2 value becomes the new x1, that is, x1 5 1.528. Further, we do not have to recalculate f(x1) because it was determined on the previous it- eration as f(1.528) 5 1.765. All that remains is to compute the new values of d and x2,

d 5 15 2 1

2 (2.472 2 0) 5 1.528

x2 5 2.4721 2 1.528 5 0.944

The function evaluation at x2 is f(0.994) 5 1.531. Since this value is less than the function value at x1, the maximum is in the interval prescribed by x2, x1, and xu. The process can be repeated, with the results tabulated below:

i xl f(xl) x2 f(x2) x1 f(x1) xu f(xu) d

1 0 0 1.5279 1.7647 2.4721 0.6300 4.0000 23.1136 2.4721 2 0 0 0.9443 1.5310 1.5279 1.7647 2.4721 0.6300 1.5279 3 0.9443 1.5310 1.5279 1.7647 1.8885 1.5432 2.4721 0.6300 0.9443 4 0.9443 1.5310 1.3050 1.7595 1.5279 1.7647 1.8885 1.5432 0.5836 5 1.3050 1.7595 1.5279 1.7647 1.6656 1.7136 1.8885 1.5432 0.3607 6 1.3050 1.7595 1.4427 1.7755 1.5279 1.7647 1.6656 1.7136 0.2229 7 1.3050 1.7595 1.3901 1.7742 1.4427 1.7755 1.5279 1.7647 0.1378 8 1.3901 1.7742 1.4427 1.7755 1.4752 1.7732 1.5279 1.7647 0.0851

cha9792x_ch13_355-369.indd Page 360 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 360 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.1 GOLDEN-SECTION SEARCH 361

Note that the current maximum is highlighted for every iteration. After the eighth iteration, the maximum occurs at x 5 1.4427 with a function value of 1.7755. Thus, the result is converging on the true value of 1.7757 at x 5 1.4276.

Recall that for bisection (Sec. 5.2.1), an exact upper bound for the error can be cal- culated at each iteration. Using similar reasoning, an upper bound for golden-section search can be derived as follows: Once an iteration is complete, the optimum will either fall in one of two intervals. If x2 is the optimum function value, it will be in the lower interval (xl, x2, x1). If x1 is the optimum function value, it will be in the upper interval (x2, x1, xu). Because the interior points are symmetrical, either case can be used to defi ne the error. Looking at the upper interval, if the true value were at the far left, the maximum distance from the estimate would be

¢xa 5 x1 2 x2 5 xl 1 R(xu 2 xl) 2 xu 1 R(xu 2 xl)

5 (xl 2 xu) 1 2R(xu 2 xl)

5 (2R 2 1)(xu 2 xl)

or 0.236(xu 2 xl). If the true value were at the far right, the maximum distance from the estimate would be

¢xb 5 xu 2 x1 5 xu 2 xl 2 R(xu 2 xl)

5 (1 2 R)(xu 2 xl)

or 0.382(xu 2 xl). Therefore, this case would represent the maximum error. This result can then be normalized to the optimal value for that iteration, xopt, to yield

ea 5 (1 2 R) ` xu 2 xlxopt ` 100% This estimate provides a basis for terminating the iterations. Pseudocode for the golden-section-search algorithm for maximization is presented in Fig. 13.5a. The minor modifi cations to convert the algorithm to minimization are listed in Fig. 13.5b. In both versions the x value for the optimum is returned as the function value (gold). In addition, the value of f(x) at the optimum is returned as the variable ( fx). You may be wondering why we have stressed the reduced function evaluations of the golden-section search. Of course, for solving a single optimization, the speed savings would be negligible. However, there are two important contexts where minimizing the number of function evaluations can be important. These are

1. Many evaluations. There are cases where the golden-section-search algorithm may be a part of a much larger calculation. In such cases, it may be called many times. Therefore, keeping function evaluations to a minimum could pay great dividends for such cases.

cha9792x_ch13_355-369.indd Page 361 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 361 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

362 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

FUNCTION Gold (xlow, xhigh, maxit, es, fx) R 5 (50.5 2 1)Y2 x/ = xlow; xu 5 xhigh iter 5 1 d 5 R * (xu 2 x/) x1 5 x/ 1 d; x2 5 xu 2 d f1 5 f(x1) f2 5 f(x2) IF f1 . f2 THEN IF f1 , f2 THEN xopt 5 x1 fx 5 f1 ELSE xopt 5 x2 fx 5 f2 END IF DO d 5 R*d; xint 5 xu 2 x/ IF f1 . f2 THEN IF f1 , f2 THEN x/ 5 x2 x2 5 x1 x1 5 x/1d f2 5 f1 f1 5 f(x1) ELSE xu 5 x1 x1 5 x2 x2 5 xu2d f1 5 f2 f2 5 f(x2) END IF iter 5 iter11 IF f1 . f2 THEN IF f1 , f2 THEN xopt 5 x1 fx 5 f1 ELSE xopt 5 x2 fx 5 f2 END IF IF xopt fi 0. THEN ea 5 (1.2R) *ABS(xintyxopt) * 100. END IF IF ea # es OR iter $ maxit EXIT END DO Gold 5 xopt END Gold (a) Maximization (b) Minimization

FIGURE 13.5 Algorithm for the golden- section search.

cha9792x_ch13_355-369.indd Page 362 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 362 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.2 PARABOLIC INTERPOLATION 363

2. Time-consuming evaluation. For pedagogical reasons, we use simple functions in most of our examples. You should understand that a function can be very complex and time- consuming to evaluate. For example, in a later part of this book, we will describe how optimization can be used to estimate the parameters of a model consisting of a system of differential equations. For such cases, the “function” involves time-consuming model integration. Any method that minimizes such evaluations would be advantageous.

13.2 PARABOLIC INTERPOLATION

Parabolic interpolation takes advantage of the fact that a second-order polynomial often provides a good approximation to the shape of f(x) near an optimum (Fig. 13.6). Just as there is only one straight line connecting two points, there is only one qua- dratic polynomial or parabola connecting three points. Thus, if we have three points that jointly bracket an optimum, we can fi t a parabola to the points. Then we can differenti- ate it, set the result equal to zero, and solve for an estimate of the optimal x. It can be shown through some algebraic manipulations that the result is

x3 5 f(x0) (x21 2 x22) 1 f(x1) (x22 2 x20) 1 f(x2) (x20 2 x21)

2 f(x0) (x1 2 x2) 1 2 f(x1) (x2 2 x0) 1 2 f(x2) (x0 2 x1) (13.7)

where x0, x1, and x2 are the initial guesses, and x3 is the value of x that corresponds to the maximum value of the parabolic fi t to the guesses. After generating the new point, there are two strategies for selecting the points for the next iteration. The simplest ap- proach, which is similar to the secant method, is to merely assign the new points se- quentially. That is, for the new iteration, z0 5 z1, z1 5 z2, and z2 5 z3. Alternatively, as illustrated in the following example, a bracketing approach, similar to bisection or the golden-section search, can be employed.

FIGURE 13.6 Graphical description of parabolic interpolation.

Parabolic approximation of maximum

Parabolic function

True maximum

True functionf (x)

xx0 x1 x3 x2

cha9792x_ch13_355-369.indd Page 363 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 363 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

364 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

EXAMPLE 13.2 Parabolic Interpolation

Problem Statement. Use parabolic interpolation to approximate the maximum of

f(x) 5 2 sin x 2 x2

10

with initial guesses of x0 5 0, x1 5 1, and x2 5 4.

Solution. The function values at the three guesses can be evaluated,

x0 5 0  f(x0) 5 0

x1 5 1  f(x1) 5 1.5829

x2 5 4  f(x2) 5 23.1136

and substituted into Eq. (13.7) to give

x3 5 0(12 2 42) 1 1.5829(42 2 02) 1 (23.1136)(02 2 12)

2(0)(1 2 4) 1 2(1.5829)(4 2 0) 1 2(23.1136)(0 2 1) 5 1.5055

which has a function value of f(1.5055) 5 1.7691. Next, a strategy similar to the golden-section search can be employed to determine which point should be discarded. Because the function value for the new point is higher than for the intermediate point (x1) and the new x value is to the right of the intermedi- ate point, the lower guess (x0) is discarded. Therefore, for the next iteration,

x0 5 1   f(x0) 5 1.5829

x1 5 1.5055   f(x1) 5 1.7691

x2 5 4   f(x2) 5 23.1136

which can be substituted into Eq. (13.7) to give

x3 5 1.5829(1.50552 2 42) 1 1.7691(42 2 12) 1 (23.1136)(12 2 1.50552)

2(1.5829)(1.5055 2 4) 1 2(1.7691)(4 2 1) 1 2(23.1136)(1 2 1.5055)

5 1.4903

which has a function value of f(1.4903) 5 1.7714. The process can be repeated, with the results tabulated below:

i x0 f(x0) x1 f(x1) x2 f(x2) x3 f(x3)

1 0.0000 0.0000 1.0000 1.5829 4.0000 23.1136 1.5055 1.7691 2 1.0000 1.5829 1.5055 1.7691 4.0000 23.1136 1.4903 1.7714 3 1.0000 1.5829 1.4903 1.7714 1.5055 1.7691 1.4256 1.7757 4 1.0000 1.5829 1.4256 1.7757 1.4903 1.7714 1.4266 1.7757 5 1.4256 1.7757 1.4266 1.7757 1.4903 1.7714 1.4275 1.7757

Thus, within fi ve iterations, the result is converging rapidly on the true value of 1.7757 at x 5 1.4276.

cha9792x_ch13_355-369.indd Page 364 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 364 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.3 NEWTON’S METHOD 365

We should mention that just like the false-position method, parabolic interpolation can get hung up with just one end of the interval converging. Thus, convergence can be slow. For example, notice that in our example, 1.0000 was an endpoint for most of the iterations. This method, as well as others using third-order polynomials, can be formulated into algorithms that contain convergence tests, careful selection strategies for the points to retain on each iteration, and attempts to minimize round-off error accumulation.

13.3 NEWTON’S METHOD

Recall that the Newton-Raphson method of Chap. 6 is an open method that fi nds the root x of a function such that f(x) 5 0. The method is summarized as

xi11 5 xi 2 f(xi)

f  ¿(xi)

A similar open approach can be used to fi nd an optimum of f(x) by defi ning a new function, g(x) 5 f 9(x). Thus, because the same optimal value x* satisfi es both

f  ¿(x*) 5 g(x*) 5 0

we can use the following,

xi11 5 xi 2 f  ¿(xi) f  –(xi)

(13.8)

as a technique to fi nd the minimum or maximum of f(x). It should be noted that this equation can also be derived by writing a second-order Taylor series for f(x) and setting the derivative of the series equal to zero. Newton’s method is an open method similar to Newton-Raphson because it does not require initial guesses that bracket the optimum. In addition, it also shares the disadvantage that it may be divergent. Finally, it is usually a good idea to check that the second derivative has the correct sign to confi rm that the technique is converging on the result you desire.

EXAMPLE 13.3 Newton’s Method

Problem Statement. Use Newton’s method to fi nd the maximum of

f(x) 5 2 sin x 2 x2

10

with an initial guess of x0 5 2.5.

Solution. The fi rst and second derivatives of the function can be evaluated as

f  ¿(x) 5 2 cos x 2 x

5

f –(x) 5 22 sin x 2 1

5

cha9792x_ch13_355-369.indd Page 365 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 365 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

366 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

which can be substituted into Eq. (13.8) to give

xi11 5 xi 2 2 cos xi 2 xiy5

22 sin xi 2 1y5

Substituting the initial guess yields

x1 5 2.5 2 2 cos 2.5 2 2.5y5 22 sin 2.5 2 1y5

5 0.99508

which has a function value of 1.57859. The second iteration gives

x1 5 0.995 2 2 cos 0.995 2 0.995y5

22 sin 0.995 2 1y5 5 1.46901

which has a function value of 1.77385. The process can be repeated, with the results tabulated below:

i x f(x) f’(x) f’’(x)

0 2.5 0.57194 22.10229 21.39694 1 0.99508 1.57859 0.88985 21.87761 2 1.46901 1.77385 20.09058 22.18965 3 1.42764 1.77573 20.00020 22.17954 4 1.42755 1.77573 0.00000 22.17952

Thus, within four iterations, the result converges rapidly on the true value.

Although Newton’s method works well in some cases, it is impractical for cases where the derivatives cannot be conveniently evaluated. For these cases, other approaches that do not involve derivative evaluation are available. For example, a secant-like version of Newton’s method can be developed by using fi nite-difference approximations for the derivative evaluations. A bigger reservation regarding the approach is that it may diverge based on the nature of the function and the quality of the initial guess. Thus, it is usually employed only when we are close to the optimum. As described next, hybrid techniques that use bracketing approaches far from the optimum and open methods near the optimum attempt to exploit the strong points of both approaches.

13.4 BRENT’S METHOD

Recall that in Sec. 6.4, we described Brent’s method for root location. This hybrid method combined several root-fi nding methods into a single algorithm that balanced reliability with effi ciency. Brent also developed a similar approach for one-dimensional minimization. It combines the slow, dependable golden-section search with the faster, but possibly unreliable, parabolic interpolation. It fi rst attempts parabolic interpolation and keeps applying it as long as ac- ceptable results are obtained. If not, it uses the golden-section search to get matters in hand. Figure 13.7 presents pseudocode for the algorithm based on a MATLAB software M-fi le developed by Cleve Moler (2005). It represents a stripped-down version of the

cha9792x_ch13_355-369.indd Page 366 09/12/13 9:11 AM F-468 cha9792x_ch13_355-369.indd Page 366 09/12/13 9:11 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

13.4 BRENT’S METHOD 367

Function fminsimp(x1, xu) tol 5 0.000001; phi 5 (1 + 15)/2;; rho 5 2 2 phi u 5 x1 1 rho*(xu 2 x1); v 5 u; w 5 u; x 5 u fu 5 f(u); fv 5 fu; fw 5 fu; fx 5 fu xm 5 0.5*(x1 1 xu); d 5 0; e 5 0 DO IF |x 2 xm| # tol EXIT para 5 |e| . tol IF para THEN (Try parabolic fit) r 5 (x 2 w)*(fx 2 fv); q 5 (x 2 v)*(fx 2 fw) p 5 (x 2 v)*q 2 (x 2 w)*r; s 5 2*(q 2 r) IF s . 0 THEN p 5 2p s 5 |s| ' Is the parabola acceptable? para 5 |p| , |0.5*s*e| And p . s*(x1 2 x) And p , s*(xu 2 x) IF para THEN e 5 d; d 5 p/s (Parabolic interpolation step) ENDIF ENDIF IF Not para THEN IF x $ xm THEN (Golden-section search step) e 5 x1 2 x ELSE e 5 xu 2 x ENDIF d 5 rho*e ENDIF u 5 x 1 d; fu 5 f(u) IF fu # fx THEN (Update x1, xu, x, v, w, xm) IF u $ x THEN x1 5 x ELSE xu 5 x ENDIF v 5 w; fv 5 fw; w 5 x; fw 5 fx; x 5 u; fx 5 fu ELSE IF u , x THEN x1 5 u ELSE xu 5 u ENDIF IF fu # fw Or w 5 x THEN v 5 w; fv 5 fw; w 5 u; fw 5 fu ELSEIF fu # fv Or v 5 x Or v 5 w THEN v 5 u; fv 5 fu ENDIF ENDIF xm 5 0.5*(x1 1 xu) ENDDO fminsimp 5 fu END fminsimp

FIGURE 13.7 Pseudocode for Brent’s minimum-fi nding algorithm based on a MATLAB M-fi le developed by Cleve Moler (2005).

cha9792x_ch13_355-369.indd Page 367 09/12/13 9:11 AM F-468 cha9792x_ch13_355-369.indd Page 367 09/12/13 9:11 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

368 ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION

fminbnd function, which is the professional minimization function employed in MATLAB. For that reason, we call the simplifi ed version fminsimp. Note that it requires another function f that holds the equation for which the minimum is being evaluated. This concludes our treatment of methods to solve the optima of functions of a single variable. Some engineering examples are presented in Chap. 16. In addition, the techniques described here are an important element of some procedures to optimize multivariable functions, as discussed in Chap. 14.

PROBLEMS

13.1 Given the formula

f (x) 5 2x2 1 8x 2 12

(a) Determine the maximum and the corresponding value of x for this function analytically (i.e., using differentiation).

(b) Verify that Eq. (13.7) yields the same results based on initial guesses of x0 5 0, x1 5 2, and x2 5 6.

13.2 Given

f (x) 5 21.5x6 2 2x4 1 12x

(a) Plot the function. (b) Use analytical methods to prove that the function is concave for

all values of x. (c) Differentiate the function and then use a root-location

method to solve for the maximum f(x) and the corresponding value of x.

13.3 Solve for the value of x that maximizes f(x) in Prob. 13.2 using the golden-section search. Employ initial guesses of xl 5 0 and xu 5 2 and perform three iterations. 13.4 Repeat Prob. 13.3, except use parabolic interpolation in the same fashion as Example 13.2. Employ initial guesses of x0 5 0, x1 5 1, and x2 5 2 and perform three iterations. 13.5 Repeat Prob. 13.3 but use Newton’s method. Employ an ini- tial guess of x0 5 2 and perform three iterations. 13.6 Employ the following methods to fi nd the maximum of

f (x) 5 4x 2 1.8x2 1 1.2x3 2 0.3x4

(a) Golden-section search (xl 5 22, xu 5 4, es 5 1%). (b) Parabolic interpolation (x0 5 1.75, x1 5 2, x2 5 2.5, itera-

tions 5 4). Select new points sequentially as in the secant method.

(c) Newton’s method (x0 5 3, es 5 1%). 13.7 Consider the following function:

f (x) 5 2 x4 2 2x3 2 8x2 2 5x

Use analytical and graphical methods to show the function has a maximum for some value of x in the range 22 # x # 1.

13.8 Employ the following methods to fi nd the maximum of the function from Prob. 13.7: (a) Golden-section search (xl 5 22, xu 5 1, es 5 1%). (b) Parabolic interpolation (x0 5 22, x1 5 21, x2 5 1, itera-

tions 5 4). Select new points sequentially as in the secant method.

(c) Newton’s method (x0 5 21, es 5 1%). 13.9 Consider the following function:

f (x) 5 2x 1 3 x

Perform 10 iterations of parabolic interpolation to locate the mini- mum. Select new points in the same fashion as in Example 13.2. Comment on the convergence of your results. (x0 5 0.1, x1 5 0.5, x2 5 5) 13.10 Consider the following function:

f (x) 5 3 1 6x 1 5x2 1 3x3 1 4x4

Locate the minimum by fi nding the root of the derivative of this function. Use bisection with initial guesses of xl 5 22 and xu 5 1. 13.11 Determine the minimum of the function from Prob. 13.10 with the following methods: (a) Newton’s method (x0 5 21, es 5 1%). (b) Newton’s method, but using a fi nite difference approximation

for the derivative estimates.

f ¿(x) 5 f (xi 1 dxi) 2 f (xi 2 dxi)

2dxi

f –(x) 5 f (xi 1 dxi) 2 2f (xi) 2 f (xi 2 dxi)

(dxi) 2

where d 5 a perturbation fraction (5 0.01). Use an initial guess of x0 5 21 and iterate to es 5 1%. 13.12 Develop a program using a programming or macro language to implement the golden-section search algorithm. Design the pro- gram so that it is expressly designed to locate a maximum. The subroutine should have the following features:

cha9792x_ch13_355-369.indd Page 368 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 368 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 369

Given that L 5 600 cm, E 5 50,000 kN/cm2, I 5 30,000 cm4, and w0  5 2.5 kN/cm, determine the point of maximum defl ection (a) graphically, (b) using the golden-section search until the approximate error falls below es 5 1% with initial guesses of xl 5 0 and xu 5 L. 13.19 An object with a mass of 100 kg is projected upward from the surface of the earth at a velocity of 50 m/s. If the object is subject to linear drag (c 5 15 kg/s), use the golden-section search to determine the maximum height the object attains. Hint: recall Sec. PT4.1.2. 13.20 The normal distribution is a bell-shaped curve defi ned by

y 5 e2x 2

Use the golden-section search to determine the location of the infl ection point of this curve for positive x. 13.21 An object can be projected upward at a specifi ed velocity. If it is subject to linear drag, its altitude as a function of time can be computed as

z 5 z0 1 m

c ay0 1 mgc b (1 2 e2(cym)t) 2

mg

c t

where z 5 altitude (m) above the earth’s surface (defi ned as z 5 0), z0 5 the initial altitude (m), m 5 mass (kg), c 5 a linear drag coef- fi cient (kg/s), v0 5 initial velocity (m/s), and t 5 time (s). Note that for this formulation, positive velocity is considered to be in the up- ward direction. Given the following parameter values: g 5 9.81 m/s2, z0 5 100 m, v0 5 55 m/s, m 5 80 kg, and c 5 15 kg/s, the equation can be used to calculate the jumper’s altitude. Determine the time and altitude of the peak elevation (a) graphically, (b) analytically, and (c) with the golden-section search until the approximate error falls be- low es 5 1% with initial guesses of tl 5 0 and tu 5 10 s. 13.22 Use the golden-section search to determine the length of the shortest ladder that reaches from the ground over the fence to touch the building’s wall (Fig. P13.22). Test it for the case where h 5 d 5 4 m.

• Iterate until the relative error falls below a stopping criterion or exceeds a maximum number of iterations.

• Return both the optimal x and f(x). • Minimize the number of function evaluations.

Test your program with the same problem as Example 13.1. 13.13 Develop a program as described in Prob. 13.12, but make it perform minimization or maximization depending on the user’s preference. 13.14 Develop a program using a programming or macro language to implement the parabolic interpolation algorithm. Design the pro- gram so that it is expressly designed to locate a maximum and se- lects new points as in Example 13.2. The subroutine should have the following features:

• Base it on two initial guesses, and have the program generate the third initial value at the midpoint of the interval.

• Check whether the guesses bracket a maximum. If not, the sub- routine should not implement the algorithm, but should return an error message.

• Iterate until the relative error falls below a stopping criterion or exceeds a maximum number of iterations.

• Return both the optimal x and f(x). • Minimize the number of function evaluations.

Test your program with the same problem as Example 13.2. 13.15 Develop a program using a programming or macro language to implement Newton’s method. The subroutine should have the following features:

• Iterate until the relative error falls below a stopping criterion or exceeds a maximum number of iterations.

• Returns both the optimal x and f(x).

Test your program with the same problem as Example 13.3. 13.16 Pressure measurements are taken at certain points behind an airfoil over time. These data best fi t the curve y 5 6 cos x 2 1.5 sin x from x 5 0 to 6 s. Use four iterations of the golden-search method to fi nd the minimum pressure. Set xl 5 2 and xu 5 4. 13.17 The trajectory of a ball can be computed with

y 5 (tan u0)x 2 g

2y20 cos 2 u0

x2 1 y0

where y 5 the height (m), u0 5 the initial angle (radians), y0 5 the initial velocity (m/s), g 5 the gravitational constant 5 9.81 m/s2, and y0 5 the initial height (m). Use the golden-section search to determine the maximum height given y0 5 1 m, y0 5 25 m/s and u0 5 508. Iterate until the approximate error falls below es 5 1% using initial guesses of xl 5 0 and xu 5 60 m. 13.18 The defl ection of a uniform beam subject to a linearly in- creasing distributed load can be computed as

y 5 w0

120EIL (2x5 1 2L2 x3 2 L4x)

d

h

FIGURE P13.22 A ladder leaning against a fence and just touching a wall.

cha9792x_ch13_355-369.indd Page 369 23/10/13 5:45 PM F-468 cha9792x_ch13_355-369.indd Page 369 23/10/13 5:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14 C H A P T E R 14

370

Multidimensional Unconstrained Optimization

This chapter describes techniques to fi nd the minimum or maximum of a function of several variables. Recall from Chap. 13 that our visual image of a one-dimensional search was like a roller coaster. For two-dimensional cases, the image becomes that of moun- tains and valleys (Fig. 14.1). For higher-dimensional problems, convenient images are not possible. We have chosen to limit this chapter to the two-dimensional case. We have adopted this approach because the essential features of multidimensional searches are often best communicated visually. Techniques for multidimensional unconstrained optimization can be classifi ed in a number of ways. For purposes of the present discussion, we will divide them depending on whether they require derivative evaluation. The approaches that do not require de- rivative evaluation are called nongradient, or direct, methods. Those that require deriva- tives are called gradient, or descent (or ascent), methods.

FIGURE 14.1 The most tangible way to visual- ize two-dimensional searches is in the context of ascending a mountain (maximization) or descending into a valley ( minimization). (a) A 2-D topographic map that corresponds to the 3-D mountain in (b).

Lines of constant f x

x

y

f

y

(a) (b)

cha9792x_ch14_370_389.indd Page 370 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 370 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.1 DIRECT METHODS 371

14.1 DIRECT METHODS

These methods vary from simple brute force approaches to more elegant techniques that attempt to exploit the nature of the function. We will start our discussion with a brute force approach.

14.1.1 Random Search

A simple example of a brute force approach is the random search method. As the name implies, this method repeatedly evaluates the function at randomly selected values of the independent variables. If a suffi cient number of samples are conducted, the optimum will eventually be located.

EXAMPLE 14.1 Random Search Method

Problem Statement. Use a random number generator to locate the maximum of

f(x, y) 5 y 2 x 2 2x2 2 2xy 2 y2 (E14.1.1)

in the domain bounded by x 5 22 to 2 and y 5 1 to 3. The domain is depicted in Fig. 14.2. Notice that a single maximum of 1.5 occurs at x 5 21 and y 5 1.5.

Solution. Random number generators typically generate values between 0 and 1. If we designate such a number as r, the following formula can be used to generate x values randomly within a range between xl to xu:

x 5 xl 1 (xu 2 xl)r

For the present application, xl 5 22 and xu 5 2, and the formula is

x 5 22 1 (2 2 (22))r 5 22 1 4r

This can be tested by substituting 0 and 1 to yield 22 and 2, respectively.

FIGURE 14.2 Equation (E14.1.1) showing the maximum at x 5 21 and y 5 1.5.

210

0

0 – 10

– 20

Maximum

– 1– 2 1

2

3

y

x

cha9792x_ch14_370_389.indd Page 371 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 371 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

372 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

This simple brute force approach works even for discontinuous and nondifferentiable functions. Furthermore, it always fi nds the global optimum rather than a local optimum. Its major shortcoming is that as the number of independent variables grows, the imple- mentation effort required can become onerous. In addition, it is not effi cient because it takes no account of the behavior of the underlying function. The remainder of the ap- proaches described in this chapter do take function behavior into account as well as the results of previous trials to improve the speed of convergence. Thus, although the random search can certainly prove useful in specifi c problem contexts, the following methods have more general utility and almost always lead to more effi cient convergence.

Iterations x y f (x, y)

1000 20.9886 1.4282 1.2462 2000 21.0040 1.4724 1.2490 3000 21.0040 1.4724 1.2490 4000 21.0040 1.4724 1.2490 5000 21.0040 1.4724 1.2490 6000 20.9837 1.4936 1.2496 7000 20.9960 1.5079 1.2498 8000 20.9960 1.5079 1.2498 9000 20.9960 1.5079 1.2498 10000 20.9978 1.5039 1.2500

Similarly for y, a formula for the present example could be developed as

y 5 yl 1 ( yu 2 yl)r 5 1 1 (3 2 1)r 5 1 1 2r

The following Excel VBA macrocode uses the VBA random number function Rnd, to generate (x, y) pairs. These are then substituted into Eq. (E14.1.1). The maximum value from among these random trials is stored in the variable maxf, and the correspond- ing x and y values in maxx and maxy, respectively.

maxf = −1E9 For j = 1 To n x = −2 + 4 * Rnd y = 1 + 2 * Rnd fn = y − x − 2 * x ^ 2 − 2 * x * y − y ^ 2 If fn > maxf Then maxf = fn maxx = x maxy = y End If Next j

A number of iterations yields

The results indicate that the technique homes in on the true maximum.

cha9792x_ch14_370_389.indd Page 372 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 372 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.1 DIRECT METHODS 373

It should be noted that more sophisticated search techniques are available. These are heuristic approaches that were developed to handle either nonlinear and/or discontinuous problems that classical optimization cannot usually handle well, if at all. Simulated an- nealing, tabu search, artifi cial neural networks, and genetic algorithms are a few. The most widely applied is the genetic algorithm, with a number of commercial packages available. Holland (1975) pioneered the genetic algorithm approach and Davis (1991) and Goldberg (1989) provide good overviews of the theory and application of the method.

14.1.2 Univariate and Pattern Searches

It is very appealing to have an effi cient optimization approach that does not require evaluation of derivatives. The random search method described above does not require derivative evaluation, but it is not very effi cient. This section describes an approach, the univariate search method, that is more effi cient and still does not require derivative evaluation. The basic strategy underlying the univariate search method is to change one variable at a time to improve the approximation while the other variables are held constant. Since only one variable is changed, the problem reduces to a sequence of one-dimensional searches that can be solved using a variety of methods (including those described in Chap. 13). Let us perform a univariate search graphically, as shown in Fig. 14.3. Start at point 1, and move along the x axis with y constant to the maximum at point 2. You can see that point 2 is a maximum by noticing that the trajectory along the x axis just touches a contour line at the point. Next, move along the y axis with x constant to point 3. Continue this process generating points 4, 5, 6, etc.

FIGURE 14.3 A graphical depiction of how a univariate search is conducted.

6

4

5

3

1

2

y

x

cha9792x_ch14_370_389.indd Page 373 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 373 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

374 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

Although we are gradually moving toward the maximum, the search becomes less effi cient as we move along the narrow ridge toward the maximum. However, also note that lines joining alternate points such as 1-3, 3-5 or 2-4, 4-6 point in the general direc- tion of the maximum. These trajectories present an opportunity to shoot directly along the ridge toward the maximum. Such trajectories are called pattern directions. Formal algorithms are available that capitalize on the idea of pattern directions to fi nd optimum values effi ciently. The best known of these algorithms is called Powell’s method. It is based on the observation (see Fig. 14.4) that if points 1 and 2 are obtained by one-dimensional searches in the same direction but from different starting points, then the line formed by 1 and 2 will be directed toward the maximum. Such lines are called conjugate directions. In fact, it can be proved that if f(x, y) is a quadratic function, sequential searches along conjugate directions will converge exactly in a fi nite number of steps regardless of the starting point. Since a general nonlinear function can often be reasonably ap- proximated by a quadratic function, methods based on conjugate directions are usually quite effi cient and are in fact quadratically convergent as they approach the optimum. Let us graphically implement a simplifi ed version of Powell’s method to fi nd the maximum of

f(x, y) 5 c 2 (x 2 a)2 2 (y 2 b)2

where a, b, and c are positive constants. This equation results in circular contours in the x, y plane, as shown in Fig. 14.5. Initiate the search at point 0 with starting directions h1 and h2. Note that h1 and h2 are not necessarily conjugate directions. From zero, move along h1 until a maximum is located

2

1

y

x

FIGURE 14.4 Conjugate directions.

cha9792x_ch14_370_389.indd Page 374 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 374 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 375

at point 1. Then search from point 1 along direction h2 to fi nd point 2. Next, form a new search direction h3 through points 0 and 2. Search along this direction until the maximum at point 3 is located. Then search from point 3 in the h2 direction until the maximum at point 4 is located. From point 4 arrive at point 5 by again searching along h3. Now, observe that both points 5 and 3 have been located by searching in the h3 direction from two dif- ferent points. Powell has shown that h4 (formed by points 3 and 5) and h3 are conjugate directions. Thus, searching from point 5 along h4 brings us directly to the maximum. Powell’s method can be refi ned to make it more effi cient, but the formal algorithms are beyond the scope of this text. However, it is an effi cient method that is quadratically convergent without requiring derivative evaluation.

14.2 GRADIENT METHODS

As the name implies, gradient methods explicitly use derivative information to generate effi cient algorithms to locate optima. Before describing specifi c approaches, we must fi rst review some key mathematical concepts and operations.

14.2.1 Gradients and Hessians

Recall from calculus that the fi rst derivative of a one-dimensional function provides a slope or tangent to the function being differentiated. From the standpoint of optimization, this is useful information. For example, if the slope is positive, it tells us that increasing the independent variable will lead to a higher value of the function we are exploring. From calculus, also recall that the fi rst derivative may tell us when we have reached an optimal value since this is the point that the derivative goes to zero. Further, the sign of the second derivative can tell us whether we have reached a minimum (positive second derivative) or a maximum (negative second derivative).

FIGURE 14.5 Powell’s method.

2

3

0

1

4

5

h3h2 h1

h2

h2 h3

h4

y

x

cha9792x_ch14_370_389.indd Page 375 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 375 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

376 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

These ideas were useful to us in the one-dimensional search algorithms we explored in Chap. 13. However, to fully understand multidimensional searches, we must fi rst understand how the fi rst and second derivatives are expressed in a multidimensional context.

The Gradient. Suppose we have a two-dimensional function f(x, y). An example might be your elevation on a mountain as a function of your position. Suppose that you are at a specifi c location on the mountain (a, b) and you want to know the slope in an arbitrary direction. One way to defi ne the direction is along a new axis h that forms an angle u with the x axis (Fig. 14.6). The elevation along this new axis can be thought of as a new func- tion g(h). If you defi ne your position as being the origin of this axis (that is, h 5 0), the slope in this direction would be designated as g9(0). This slope, which is called the direc- tional derivative, can be calculated from the partial derivatives along the x and y axis by

g¿(0) 5 0f 0x

cos u 1 0f 0y

sin u (14.1)

where the partial derivatives are evaluated at x 5 a and y 5 b. Assuming that your goal is to gain the most elevation with the next step, the next logical question would be: what direction is the steepest ascent? The answer to this question is provided very neatly by what is referred to mathematically as the gradient, which is defi ned as

§ f 5 0f 0x

i 1 0f 0y

j (14.2)

This vector is also referred to as “del f.” It represents the directional derivative of f(x, y) at point x 5 a and y 5 b.

x = a y = b h = 0

h

y

x

FIGURE 14.6 The directional gradient is defi ned along an axis h that forms an angle u with the x axis.

cha9792x_ch14_370_389.indd Page 376 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 376 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 377

Vector notation provides a concise means to generalize the gradient to n dimensions, as

§ f(x) 5 i 0f 0x1

(x)

0f 0x2

(x)

.

.

. 0f 0xn

(x)

y How do we use the gradient? For the mountain-climbing problem, if we are inter- ested in gaining elevation as quickly as possible, the gradient tells us what direction to move locally and how much we will gain by taking it. Note, however, that this strategy does not necessarily take us on a direct path to the summit! We will discuss these ideas in more depth later in this chapter.

EXAMPLE 14.2 Using the Gradient to Evaluate the Path of Steepest Ascent

Problem Statement. Employ the gradient to evaluate the steepest ascent direction for the function

f(x, y) 5 xy2

at the point (2, 2). Assume that positive x is pointed east and positive y is pointed north.

Solution. First, our elevation can be determined as

f(2, 2) 5 2(2)2 5 8

Next, the partial derivatives can be evaluated,

0f 0x

5 y2 5 22 5 4

0f 0y

5 2xy 5 2(2)(2) 5 8

which can be used to determine the gradient as

§ f 5 4i 1 8j

This vector can be sketched on a topographical map of the function, as in Fig. 14.7. This immediately tells us that the direction we must take is

u 5 tan21 a8 4 b 5 1.107 radians (563.4°)

relative to the x axis. The slope in this direction, which is the magnitude of = f, can be calculated as

242 1 82 5 8.944

cha9792x_ch14_370_389.indd Page 377 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 377 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

378 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

Thus, during our fi rst step, we will initially gain 8.944 units of elevation rise for a unit distance advanced along this steepest path. Observe that Eq. (14.1) yields the same result,

g¿(0) 5 4 cos(1.107) 1 8 sin(1.107) 5 8.944

Note that for any other direction, say u 5 1.107y2 5 0.5235, g9(0) 5 4 cos(0.5235) 1 8 sin(0.5235) 5 7.608, which is smaller. As we move forward, both the direction and magnitude of the steepest path will change. These changes can be quantifi ed at each step using the gradient, and your climb- ing direction modifi ed accordingly. A fi nal insight can be gained by inspecting Fig. 14.7. As indicated, the direction of steepest ascent is perpendicular, or orthogonal, to the elevation contour at the coordinate (2, 2). This is a general characteristic of the gradient.

0 0

1

2

3

4

1 2 3 4

y

x

8 24 40

FIGURE 14.7 The arrow follows the direction of steepest ascent calculated with the gradient.

Aside from defi ning a steepest path, the fi rst derivative can also be used to discern whether an optimum has been reached. As is the case for a one-dimensional function, if the partial derivatives with respect to both x and y are zero, a two-dimensional optimum has been reached.

The Hessian. For one-dimensional problems, both the fi rst and second derivatives pro- vide valuable information for searching out optima. The fi rst derivative (a) provides a steepest trajectory of the function and (b) tells us that we have reached an optimum. Once at an optimum, the second derivative tells us whether we are a maximum [negative

cha9792x_ch14_370_389.indd Page 378 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 378 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 379

f 0(x)] or a minimum [positive f 0(x)]. In the previous paragraphs, we illustrated how the gradient provides best local trajectories for multidimensional problems. Now, we will examine how the second derivative is used in such contexts. You might expect that if the partial second derivatives with respect to both x and y are both negative, then you have reached a maximum. Figure 14.8 shows a function where this is not true. The point (a, b) of this graph appears to be a minimum when observed along either the x dimension or the y dimension. In both instances, the second partial derivatives are positive. However, if the function is observed along the line y 5 x, it can be seen that a maximum occurs at the same point. This shape is called a saddle, and clearly, neither a maximum or a minimum occurs at the point. Whether a maximum or a minimum occurs involves not only the partials with respect to x and y but also the second partial with respect to x and y. Assuming that the partial derivatives are continuous at and near the point being evaluated, the following quantity can be computed:

ZH Z 5 02 f 0 x 2

02 f 0 y2

2 a 02 f 0 x 0 y

b2 (14.3) Three cases can occur

If Z H Z . 0 and 02fy0x2 . 0, then f(x, y) has a local minimum. If Z H Z . 0 and 02fy0x2 , 0, then f(x, y) has a local maximum. If Z H Z , 0, then f(x, y) has a saddle point.

f (x, y)

(a, b)

x

y y = x

FIGURE 14.8 A saddle point (x 5 a and y 5 b). Notice that when the curve is viewed along the x and y directions, the function appears to go through a minimum (positive second derivative), whereas when viewed along an axis x 5 y, it is concave downward (negative second derivative).

cha9792x_ch14_370_389.indd Page 379 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 379 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

380 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

The quantity Z H Z is equal to the determinant of a matrix made up of the second derivatives,1

H 5 ≥ 02 f 0 x 2

02 f 0 x 0 y

02 f 0 y 0 x

02 f 0 y 2

¥ (14.4)

where this matrix is formally referred to as the Hessian of f. Besides providing a way to discern whether a multidimensional function has reached an optimum, the Hessian has other uses in optimization (for example, for the multidi- mensional form of Newton’s method). In particular, it allows searches to include second- order curvature to attain superior results.

Finite-Difference Approximations. It should be mentioned that, for cases where they are diffi cult or inconvenient to compute analytically, both the gradient and the determi- nant of the Hessian can be evaluated numerically. In most cases, the approach introduced in Sec. 6.3.3 for the modifi ed secant method is employed. That is, the independent variables can be perturbed slightly to generate the required partial derivatives. For ex- ample, if a centered-difference approach is adopted, they can be computed as

0f 0x

5 f(x 1 dx, y) 2 f(x 2 dx, y)

2dx (14.5)

0f 0y

5 f(x, y 1 dy) 2 f(x, y 2 dy)

2dy (14.6)

02 f 0x2

5 f(x 1 dx, y) 2 2 f(x, y) 1 f(x 2 dx, y)

dx2 (14.7)

02 f 0y2

5 f(x, y 1 dy) 2 2 f(x, y) 1 f(x, y 2 dy)

dy2 (14.8)

02 f 0x0y

5

f(x 1 dx, y 1 dy) 2 f(x 1 dx, y 2 dy) 2 f(x 2 dx, y 1 dy) 1 f(x 2 dx, y 2 dy)

4dx dy

(14.9)

where d is some small fractional value. Note that the methods employed in commercial software packages also use forward differences. In addition, they are usually more complicated than the approximations listed in Eqs. (14.5) through (14.9). Dennis and Schnabel (1996) provide more detail on such approaches. Regardless of how the approximation is implemented, the important point is that you may have the option of evaluating the gradient and/or the Hessian analytically. This can sometimes be an arduous task, but the performance of the algorithm may benefi t

1Note that 02fy(0x0y) 5 02fy(0y0x).

cha9792x_ch14_370_389.indd Page 380 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 380 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 381

enough to make your effort worthwhile. The closed-form derivatives will be exact, but more importantly, you will reduce the number of function evaluations. This latter point can have a critical impact on the execution time. On the other hand, you will often exercise the option of having the quantities com- puted internally using numerical approaches. In many cases, the performance will be quite adequate and you will be saved the diffi culty of numerous partial differentiations. Such would be the case on the optimizers used in certain spreadsheets and mathematical software packages (for example, Excel). In such cases, you may not even be given the option of entering an analytically derived gradient and Hessian. However, for small to moderately sized problems, this is usually not a major shortcoming.

14.2.2 Steepest Ascent Method

An obvious strategy for climbing a hill would be to determine the maximum slope at your starting position and then start walking in that direction. But clearly, another prob- lem arises almost immediately. Unless you were really lucky and started on a ridge that pointed directly to the summit, as soon as you moved, your path would diverge from the steepest ascent direction. Recognizing this fact, you might adopt the following strategy. You could walk a short distance along the gradient direction. Then you could stop, reevaluate the gradient and walk another short distance. By repeating the process you would eventually get to the top of the hill. Although this strategy sounds superfi cially sound, it is not very practical. In par- ticular, the continuous reevaluation of the gradient can be computationally demanding. A preferred approach involves moving in a fi xed path along the initial gradient until f(x, y) stops increasing, that is, becomes level along your direction of travel. This stopping point becomes the starting point where § f is reevaluated and a new direction followed. The process is repeated until the summit is reached. This approach is called the steepest ascent method.2 It is the most straightforward of the gradient search techniques. The basic idea behind the approach is depicted in Fig. 14.9. We start at an initial point (x0, y0) labeled “0” in the fi gure. At this point, we deter- mine the direction of steepest ascent, that is, the gradient. We then search along the direction of the gradient, h0, until we fi nd a maximum, which is labeled “1” in the fi gure. The process is then repeated. Thus, the problem boils down to two parts: (1) determining the “best” direction to search and (2) determining the “best value” along that search direction. As we will see, the effectiveness of the various algorithms described in the coming pages depends on how clever we are at both parts. For the time being, the steepest ascent method uses the gradient approach as its choice for the “best” direction. We have already shown how the gradient is evaluated in Example 14.1. Now, before examining how the algorithm goes about locating the maxi- mum along the steepest direction, we must pause to explore how to transform a function of x and y into a function of h along the gradient direction.

2Because of our emphasis on maximization here, we use the terminology steepest ascent. The same approach can also be used for minimization, in which case the terminology steepest descent is used.

cha9792x_ch14_370_389.indd Page 381 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 381 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

382 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

Starting at x0, y0 the coordinates of any point in the gradient direction can be ex- pressed as

x 5 x0 1 0f 0x

h (14.10)

y 5 y0 1 0f 0y

h (14.11)

FIGURE 14.9 A graphical depiction of the method of steepest ascent.

2

1

0

h0

h2

h1

y

x

FIGURE 14.10 The relationship between an arbitrary direction h and x and y coordinates.

10

y

x

6

2

741

�f = 3i + 4j

h =

2

h =

1

h =

0

cha9792x_ch14_370_389.indd Page 382 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 382 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 383

where h is distance along the h axis. For example, suppose x0 5 1 and y0 5 2 and § f 5 3i 1 4j, as shown in Fig. 14.10. The coordinates of any point along the h axis are given by

x 5 1 1 3h (14.12)

y 5 2 1 4h (14.13)

The following example illustrates how we can use these transformations to convert a two-dimensional function of x and y into a one-dimensional function in h.

EXAMPLE 14.3 Developing a 1-D Function Along the Gradient Direction

Problem Statement. Suppose we have the following two-dimensional function:

f(x, y) 5 2xy 1 2x 2 x2 2 2y2

Develop a one-dimensional version of this equation along the gradient direction at point x 5 21 and y 5 1.

Solution. The partial derivatives can be evaluated at (21, 1),

0f 0x

5 2y 1 2 2 2x 5 2(1) 1 2 2 2(21) 5 6

0f 0y

5 2x 2 4y 5 2(21) 2 4(1) 5 26

Therefore, the gradient vector is

§ f 5 6i 2 6j

To fi nd the maximum, we could search along the gradient direction, that is, along an h axis running along the direction of this vector. The function can be expressed along this axis as

f ax0 1 0f0x h, y0 1 0f 0y

hb 5 f(21 1 6h, 1 2 6h) 5 2(21 1 6h)(1 2 6h) 1 2(21 1 6h) 2 (21 1 6h)2 2 2(1 2 6h)2

where the partial derivatives are evaluated at x 5 21 and y 5 1. By combining terms, we develop a one-dimensional function g(h) that maps f(x, y) along the h axis,

g(h) 5 2180h2 1 72h 2 7

Now that we have developed a function along the path of steepest ascent, we can explore how to answer the second question. That is, how far along this path do we travel? One approach might be to move along this path until we fi nd the maximum of this func- tion. We will call the location of this maximum h*. This is the value of the step that maximizes g (and hence, f ) in the gradient direction. This problem is equivalent to fi nd- ing the maximum of a function of a single variable h. This can be done using different one-dimensional search techniques like the ones we discussed in Chap. 13. Thus, we

cha9792x_ch14_370_389.indd Page 383 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 383 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

384 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

convert from fi nding the optimum of a two-dimensional function to performing a one- dimensional search along the gradient direction. This method is called steepest ascent when an arbitrary step size h is used. If a value of a single step h* is found that brings us directly to the maximum along the gradient direction, the method is called the optimal steepest ascent.

EXAMPLE 14.4 Optimal Steepest Ascent

Problem Statement. Maximize the following function:

f(x, y) 5 2xy 1 2x 2 x2 2 2y2

using initial guesses, x 5 21 and y 5 1.

Solution. Because this function is so simple, we can fi rst generate an analytical solu- tion. To do this, the partial derivatives can be evaluated as

0 f 0 x

5 2y 1 2 2 2x 5 0

0 f 0 y

5 2x 2 4y 5 0

This pair of equations can be solved for the optimum, x 5 2 and y 5 1. The second partial derivatives can also be determined and evaluated at the optimum,

02 f 0x2

5 22

02 f 0y2

5 24

02f 0x0y

5 02f 0y0x

5 2

and the determinant of the Hessian is computed [Eq. (14.3)],

ZH Z 5 22(24) 2 22 5 4

Therefore, because Z H Z . 0 and 02fy0x2 , 0, function value f(2, 1) is a maximum. Now let us implement steepest ascent. Recall that, at the end of Example 14.3, we had already implemented the initial steps of the problem by generating

g(h) 5 2180h2 1 72h 2 7

Now, because this is a simple parabola, we can directly locate the maximum (that is, h 5 h*) by solving the problem,

g¿(h*) 5 0 2360h* 1 72 5 0

h* 5 0.2

This means that if we travel along the h axis, g(h) reaches a minimum value when h 5 h* 5 0.2. This result can be placed back into Eqs. (14.10) and (14.11) to solve for the

cha9792x_ch14_370_389.indd Page 384 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 384 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 385

(x, y) coordinates corresponding to this point,

x 5 21 1 6(0.2) 5 0.2

y 5 1 2 6(0.2) 5 20.2

This step is depicted in Fig. 14.11 as the move from point 0 to 1. The second step is merely implemented by repeating the procedure. First, the partial derivatives can be evaluated at the new starting point (0.2, 20.2) to give

0 f 0 x

5 2(20.2) 1 2 2 2(0.2) 5 1.2

0 f 0 y

5 2(0.2) 2 4(20.2) 5 1.2

Therefore, the gradient vector is

§ f 5 1.2 i 1 1.2 j

This means that the steepest direction is now pointed up and to the right at a 458 angle with the x axis (see Fig. 14.11). The coordinates along this new h axis can now be expressed as

x 5 0.2 1 1.2h

y 5 20.2 1 1.2h

Substituting these values into the function yields

f(0.2 1 1.2h, 20.2 1 1.2h) 5 g(h) 5 21.44h2 1 2.88h 1 0.2

The step h* to take us to the maximum along the search direction can then be directly computed as

g¿(h*) 5 22.88h* 1 2.88 5 0 h* 5 1

FIGURE 14.11 The method of optimal steepest ascent.

2

2

1

0

Maximum

0– 2 – 1

0

2

1

3

y

x4

cha9792x_ch14_370_389.indd Page 385 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 385 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

386 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

It can be shown that the method of steepest descent is linearly convergent. Further, it tends to move very slowly along long, narrow ridges. This is because the new gradient at each maximum point will be perpendicular to the original direction. Thus, the technique takes many small steps criss-crossing the direct route to the summit. Hence, although it is reliable, there are other approaches that converge much more rapidly, particularly in the vicinity of an optimum. The remainder of the section is devoted to such methods.

14.2.3 Advanced Gradient Approaches

Conjugate Gradient Method (Fletcher-Reeves). In Sec. 14.1.2, we have seen how conjugate directions in Powell’s method greatly improved the effi ciency of a univariate search. In a similar manner, we can also improve the linearly convergent steepest ascent using conjugate gradients. In fact, an optimization method that makes use of conjugate gradients to defi ne search directions can be shown to be quadratically convergent. This also ensures that the method will optimize a quadratic function exactly in a fi nite num- ber of steps regardless of the starting point. Since most well-behaved functions can be approximated reasonably well by a quadratic in the vicinity of an optimum, quadratically convergent approaches are often very effi cient near an optimum. We have seen how starting with two arbitrary search directions, Powell’s method produced new conjugate search directions. This method is quadratically convergent and does not require gradient information. On the other hand, if evaluation of derivatives is practical, we can devise algorithms that combine the ideas of steepest descent and con- jugate directions to achieve robust initial performance and rapid convergence as the technique gravitates toward the optimum. The Fletcher-Reeves conjugate gradient algo- rithm modifi es the steepest-ascent method by imposing the condition that successive gradient search directions be mutually conjugate. The proof and algorithm are beyond the scope of the text but are described by Rao (1996).

Newton’s Method. Newton’s method for a single variable (recall Sec. 13.3) can be extended to multivariate cases. Write a second-order Taylor series for f(x) near x 5 xi,

f(x) 5 f(xi) 1 § f T (xi)(x 2 xi) 1 1

2 (x 2 xi)

T Hi(x 2 xi)

where Hi is the Hessian matrix. At the minimum,

0f(x) 0xj

5 0  for j 5 1, 2, p , n

This result can be placed back into Eqs. (14.10) and (14.11) to solve for the (x, y) co- ordinates corresponding to this new point,

x 5 0.2 1 1.2(1) 5 1.4

y 5 20.2 1 1.2(1) 5 1

As depicted in Fig. 14.11, we move to the new coordinates, labeled point 2 in the plot, and in so doing move closer to the maximum. The approach can be repeated with the fi nal result converging on the analytical solution, x 5 2 and y 5 1.

cha9792x_ch14_370_389.indd Page 386 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 386 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

14.2 GRADIENT METHODS 387

Thus,

§ f 5 § f(xi) 1 Hi(x 2 xi) 5 0

If H is nonsingular,

xi11 5 xi 2 H 21 i § f (14.14)

which can be shown to converge quadratically near the optimum. This method again performs better than the steepest ascent method (see Fig. 14.12). However, note that the method requires both the computation of second derivatives and matrix inversion at each iteration. Thus, the method is not very useful in practice for functions with large numbers of variables. Furthermore, Newton’s method may not converge if the starting point is not close to the optimum.

Marquardt Method. We know that the method of steepest ascent increases the func- tion value even if the starting point is far from an optimum. On the other hand, we have just described Newton’s method, which converges rapidly near the maximum. Marquardt’s method uses the steepest descent method when x is far from x*, and Newton’s method when x closes in on an optimum. This is accomplished by modifying the diagonal of the Hessian in Eq. (14.14),

H˜ i 5 Hi 1 ai I

where ai is a positive constant and I is the identity matrix. At the start of the procedure, ai is assumed to be large and

H˜ 21i < 1 ai

I

FIGURE 14.12 When the starting point is close to the optimal point, following the gradient can be ineffi cient. Newton methods attempt to search along a direct path to the optimum (solid line).

y

x

cha9792x_ch14_370_389.indd Page 387 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 387 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

388 MULTIDIMENSIONAL UNCONSTRAINED OPTIMIZATION

which reduces Eq. (14.14) to the steepest ascent method. As the iterations proceed, ai approaches zero and the method becomes Newton’s method. Thus, Marquardt’s method offers the best of both worlds: it plods along reliably from poor initial starting values yet accelerates rapidly when it approaches the optimum. Unfortunately, the method still requires Hessian evaluation and matrix inversion at each step. It should be noted that the Marquardt method is primarily used for nonlinear least- squares problems.

Quasi-Newton Methods. Quasi-Newton, or variable metric, methods seek to estimate the direct path to the optimum in a manner similar to Newton’s method. However, notice that the Hessian matrix in Eq. (14.14) is composed of the second derivatives of f that vary from step to step. Quasi-Newton methods attempt to avoid these diffi culties by approximating H with another matrix A using only fi rst partial derivatives of f. The approach involves starting with an initial approximation of H21 and updating and improv- ing it with each iteration. The methods are called quasi-Newton because we do not use the true Hessian, rather an approximation. Thus, we have two approximations at work simul- taneously: (1) the original Taylor-series approximation and (2) the Hessian approximation. There are two primary methods of this type: the Davidon-Fletcher-Powell (DFP) and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithms. They are similar except for details concerning how they handle round-off error and convergence issues. BFGS is generally recognized as being superior in most cases. Rao (1996) provides details and formal statements of both the DFP and the BFGS algorithms.

PROBLEMS

14.1 Find the directional derivative of

f (x, y) 5 x2 1 2y2

at x 5 2 and y 5 2 in the direction of h 5 2i 1 3j. 14.2 Repeat Example 14.2 for the following function at the point (0.8, 1.2).

f (x, y) 5 2xy 1 1.5y 2 1.25x2 2 2y2 1 5

14.3 Given

f (x, y) 5 2.25xy 1 1.75y 2 1.5x2 2 2y2

Construct and solve a system of linear algebraic equations that maximizes f(x). Note that this is done by setting the partial deriva- tives of f with respect to both x and y to zero. 14.4 (a) Start with an initial guess of x 5 1 and y 5 1 and apply two ap-

plications of the steepest ascent method to f(x, y) from Prob. 14.3. (b) Construct a plot from the results of (a) showing the path of the

search. 14.5 Find the gradient vector and Hessian matrix for each of the following functions: (a) f(x, y) 5 2xy2 1 3exy

(b) f(x, y, z) 5 x2 1 y2 1 2z2

(c) f(x, y) 5 ln(x2 1 2xy 1 3y2) 14.6 Find the minimum value of

f (x, y) 5 (x 2 3)2 1 (y 2 2)2

starting at x 5 1 and y 5 1, using the steepest descent method with a stopping criterion of es 5 1%. Explain your results. 14.7 Perform one iteration of the steepest ascent method to locate the maximum of

f (x, y) 5 4x 1 2y 1 x2 2 2x4 1 2xy 2 3y2

using initial guesses x 5 0 and y 5 0. Employ bisection to fi nd the optimal step size in the gradient search direction. 14.8 Perform one iteration of the optimal gradient steepest descent method to locate the minimum of

f (x, y) 5 28x 1 x2 1 12y 1 4y2 2 2xy

using initial guesses x 5 0 and y 5 0. 14.9 Develop a program using a programming or macro language to implement the random search method. Design the subprogram so that it is expressly designed to locate a maximum. Test the program with f(x, y) from Prob. 14.7. Use a range of 22 to 2 for both x and y.

cha9792x_ch14_370_389.indd Page 388 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 388 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 389

14.10 The grid search is another brute force approach to optimiza- tion. The two-dimensional version is depicted in Fig. P14.10. The x and y dimensions are divided into increments to create a grid. The function is then evaluated at each node of the grid. The denser the grid, the more likely it would be to locate the optimum.

Develop a program using a programming or macro language to implement the grid search method. Design the program so that it is expressly designed to locate a maximum. Test it with the same problem as Example 14.1. 14.11 Develop a one-dimensional equation in the pressure gradient direction at the point (4, 2). The pressure function is

f (x, y) 5 6x2y 2 9y2 2 8x2

14.12 A temperature function is

f (x, y) 5 2x3y2 2 7xy 2 x2 1 3y

Develop a one-dimensional function in the temperature gradient direction at the point (1, 1).

FIGURE P14.10 The grid search.

210

– 5 – 10 – 15 – 20 – 25

0

0

Maximum

– 1– 2 1

2

3

y

x

cha9792x_ch14_370_389.indd Page 389 23/10/13 5:42 PM F-468 cha9792x_ch14_370_389.indd Page 389 23/10/13 5:42 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15 C H A P T E R 15

390

Constrained Optimization

This chapter deals with optimization problems where constraints come into play. We fi rst discuss problems where both the objective function and the constraints are linear. For such cases, special methods are available that exploit the linearity of the underlying functions. Called linear programming methods, the resulting algorithms solve very large problems with thousands of variables and constraints with great effi ciency. They are used in a wide range of problems in engineering and management. Then we will turn briefl y to the more general problem of nonlinear constrained optimization. Finally, we provide an overview of how software packages can be employed for optimization.

15.1 LINEAR PROGRAMMING

Linear programming (LP) is an optimization approach that deals with meeting a desired objective such as maximizing profi t or minimizing cost in the presence of constraints such as limited resources. The term linear connotes that the mathematical functions representing both the objective and the constraints are linear. The term programming does not mean “computer programming,” but rather, connotes “scheduling” or “setting an agenda” (Revelle et al., 1997).

15.1.1 Standard Form

The basic linear programming problem consists of two major parts: the objective function and a set of constraints. For a maximization problem, the objective function is generally expressed as

Maximize Z 5 c1x1 1 c2 x2 1 p 1 cn xn (15.1)

where cj 5 payoff of each unit of the jth activity that is undertaken and xj 5 magnitude of the jth activity. Thus, the value of the objective function, Z, is the total payoff due to the total number of activities, n. The constraints can be represented generally as

ai1x1 1 ai2 x2 1 p 1 ain xn # bi (15.2)

cha9792x_ch15_390-415.indd Page 390 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 390 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.1 LINEAR PROGRAMMING 391

where aij 5 amount of the ith resource that is consumed for each unit of the jth activity and bi 5 amount of the ith resource that is available. That is, the resources are limited. The second general type of constraint specifi es that all activities must have a positive value,

xi $ 0 (15.3)

In the present context, this expresses the realistic notion that, for some problems, negative activity is physically impossible (for example, we cannot produce negative goods). Together, the objective function and the constraints specify the linear programming problem. They say that we are trying to maximize the payoff for a number of activities under the constraint that these activities utilize fi nite amounts of resources. Before show- ing how this result can be obtained, we will fi rst develop an example.

EXAMPLE 15.1 Setting Up the LP Problem

Problem Statement. The following problem is developed from the area of chemical or petroleum engineering. However, it is relevant to all areas of engineering that deal with producing products with limited resources. Suppose that a gas-processing plant receives a fi xed amount of raw gas each week. The raw gas is processed into two grades of heating gas, regular and premium quality. These grades of gas are in high demand (that is, they are guaranteed to sell) and yield different profi ts to the company. However, their production involves both time and on-site storage constraints. For example, only one of the grades can be produced at a time, and the facility is open for only 80 hr/week. Further, there is limited on-site storage for each of the products. All these factors are listed below (note that a metric ton, or tonne, is equal to 1000 kg):

Product

Resource Regular Premium Resource Availability

Raw gas 7 m3/tonne 11 m3/tonne 77 m3/week Production time 10 hr/tonne 8 hr/tonne 80 hr/week Storage 9 tonnes 6 tonnes

Profi t 150/tonne 175/tonne

Develop a linear programming formulation to maximize the profi ts for this operation.

Solution. The engineer operating this plant must decide how much of each gas to produce to maximize profi ts. If the amounts of regular and premium produced weekly are designated as x1 and x2, respectively, the total weekly profi t can be calculated as

Total profit 5 150x1 1 175x2

or written as a linear programming objective function,

Maximize Z 5 150x1 1 175x2

cha9792x_ch15_390-415.indd Page 391 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 391 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

392 CONSTRAINED OPTIMIZATION

The constraints can be developed in a similar fashion. For example, the total raw gas used can be computed as

Total gas used 5 7x1 1 11x2

This total cannot exceed the available supply of 77 m3/week, so the constraint can be represented as

7x1 1 11x2 # 77

The remaining constraints can be developed in a similar fashion, with the resulting total LP formulation given by

Maximize Z 5 150x1 1 175x2 (maximize profi t)

subject to

7x1 1 11x2 # 77 (material constraint)

10x1 1 8x2 # 80 (time constraint)

x1 # 9 (“regular” storage constraint)

x2 # 6 (“premium” storage constraint)

x1, x2 $ 0 (positivity constraints)

Note that the above set of equations constitute the total LP formulation. The parenthetical explanations at the right have been appended to clarify the meaning of each term.

15.1.2 Graphical Solution

Because they are limited to two or three dimensions, graphical solutions have limited practical utility. However, they are very useful for demonstrating some basic concepts that underlie the general algebraic techniques used to solve higher-dimensional problems with the computer. For a two-dimensional problem, such as the one in Example 15.1, the solution space is defi ned as a plane with x1 measured along the abscissa and x2 along the ordinate. Because they are linear, the constraints can be plotted on this plane as straight lines. If the LP prob- lem was formulated properly (that is, it has a solution), these constraint lines will delineate a region, called the feasible solution space, encompassing all possible combinations of x1 and x2 that obey the constraints and hence represent feasible solutions. The objective func- tion for a particular value of Z can then be plotted as another straight line and superimposed on this space. The value of Z can then be adjusted until it is at the maximum value while still touching the feasible space. This value of Z represents the optimal solution. The cor- responding values of x1 and x2, where Z touches the feasible solution space, represent the optimal values for the activities. The following example should help clarify the approach.

EXAMPLE 15.2 Graphical Solution

Problem Statement. Develop a graphical solution for the gas-processing problem pre- viously derived in Example 15.1:

Maximize Z 5 150x1 1 175x2

cha9792x_ch15_390-415.indd Page 392 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 392 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.1 LINEAR PROGRAMMING 393

subject to

7x1 1 11x2 # 77 (1)

10x1 1 8x2 # 80 (2)

x1 # 9 (3)

x2 # 6 (4)

x1 $ 0 (5)

x2 $ 0 (6)

We have numbered the constraints to identify them in the following graphical solution.

Solution. First, the constraints can be plotted on the solution space. For example, the fi rst constraint can be reformulated as a line by replacing the inequality by an equal sign and solving for x2:

x2 5 2 7

11 x1 1 7

Thus, as in Fig. 15.1a, the possible values of x1 and x2 that obey this constraint fall below this line (the direction designated in the plot by the small arrow). The other constraints can be evaluated similarly, as superimposed on Fig. 15.1a. Notice how they encompass a region where they are all met. This is the feasible solution space (the area ABCDE in the plot). Aside from defi ning the feasible space, Fig. 15.1a also provides additional insight. In particular, we can see that constraint 3 (storage of regular gas) is “redundant.” That is, the feasible solution space is unaffected if it were deleted.

FIGURE 15.1 Graphical solution of a linear programming problem. (a) The constraints defi ne a feasible solution space. (b) The objective function can be increased until it reaches the highest value that obeys all constraints. Graphically, the function moves up and to the right until it touches the feasible space at a single optimal point.

(b)

0

8

4 x1

x2

8

A B

C

DE

Z �

0

Z �

600

Z �

1400

(a)

0

8

4

4 x1

Redundant

4

x2

8

A

F

B

C

DE

3

6

5

1

2

cha9792x_ch15_390-415.indd Page 393 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 393 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

394 CONSTRAINED OPTIMIZATION

Next, the objective function can be added to the plot. To do this, a value of Z must be chosen. For example, for Z 5 0, the objective function becomes

0 5 150x1 1 175x2

or, solving for x2, we derive the line

x2 5 2 150

175 x1

As displayed in Fig. 15.1b, this represents a dashed line intersecting the origin. Now, since we are interested in maximizing Z, we can increase it to say, 600, and the objective function is

x2 5 600

175 2

150

175 x1

Thus, increasing the value of the objective function moves the line away from the origin. Because the line still falls within the solution space, our result is still feasible. For the same reason, however, there is still room for improvement. Hence, Z can keep increasing until a further increase will take the objective beyond the feasible region. As shown in Fig. 15.1b, the maximum value of Z corresponds to approximately 1400. At this point, x1 and x2 are equal to approximately 4.9 and 3.9, respectively. Thus, the graphical solu- tion tells us that if we produce these quantities of regular and premium, we will reap a maximum profi t of about 1400.

Aside from determining optimal values, the graphical approach provides further insights into the problem. This can be appreciated by substituting the answers back into the constraint equations,

7(4.9) 1 11(3.9) > 77 10(4.9) 1 8(3.9) > 80 4.9 # 9

3.9 # 6

Consequently, as is also clear from the plot, producing at the optimal amount of each product brings us right to the point where we just meet the resource (1) and time con- straints (2). Such constraints are said to be binding. Further, as is also evident graphically, neither of the storage constraints [(3) and (4)] acts as a limitation. Such constraints are called nonbinding. This leads to the practical conclusion that, for this case, we can increase profi ts by either increasing our resource supply (the raw gas) or increasing our production time. Further, it indicates that increasing storage would have no impact on profi t. The result obtained in the previous example is one of four possible outcomes that can be generally obtained in a linear programming problem. These are

1. Unique solution. As in the example, the maximum objective function intersects a single point.

2. Alternate solutions. Suppose that the objective function in the example had coeffi cients so that it was precisely parallel to one of the constraints. In our example problem,

cha9792x_ch15_390-415.indd Page 394 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 394 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.1 LINEAR PROGRAMMING 395

one way in which this would occur would be if the profi ts were changed to $140/ tonne and $220/tonne. Then, rather than a single point, the problem would have an infi nite number of optima corresponding to a line segment (Fig. 15.2a).

3. No feasible solution. As in Fig. 15.2b, it is possible that the problem is set up so that there is no feasible solution. This can be due to dealing with an unsolvable problem or due to errors in setting up the problem. The latter can result if the problem is over-constrained to the point that no solution can satisfy all the constraints.

4. Unbounded problems. As in Fig. 15.2c, this usually means that the problem is under- constrained and therefore open-ended. As with the no-feasible-solution case, it can often arise from errors committed during problem specifi cation.

Now let us suppose that our problem involves a unique solution. The graphical approach might suggest an enumerative strategy for hunting down the maximum. From Fig. 15.1, it should be clear that the optimum always occurs at one of the corner points where two constraints meet. Such a point is known formally as an extreme point. Thus, out of the infi nite number of possibilities in the decision space, focusing on extreme points clearly narrows down the possible options. Further, we can recognize that not every extreme point is feasible, that is, satisfying all constraints. For example, notice that point F in Fig. 15.1a is an extreme point but is not feasible. Limiting ourselves to feasible extreme points narrows the fi eld down still further. Finally, once all feasible extreme points are identifi ed, the one yielding the best value of the objective function represents the optimum solution. Finding this optimal solution could be done by exhaustively (and ineffi ciently) evaluating the value of the objective function at every feasible extreme point. The following section discusses the simplex method, which offers a preferable strategy that charts a selective course through a sequence of feasible extreme points to arrive at the optimum in an extremely effi cient manner.

FIGURE 15.2 Aside from a single optimal solution (for example, Fig. 15.1b), there are three other possible outcomes of a linear programming problem: (a) alternative optima, (b) no feasible solution, and (c) an unbounded result.

(b)

0 x1

x2

(a)

0 x1

x2

(c)

0 x1

x2

Z

cha9792x_ch15_390-415.indd Page 395 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 395 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

396 CONSTRAINED OPTIMIZATION

15.1.3 The Simplex Method

The simplex method is predicated on the assumption that the optimal solution will be an extreme point. Thus, the approach must be able to discern whether during problem solution an extreme point occurs. To do this, the constraint equations are reformulated as equalities by introducing what are called slack variables.

Slack Variables. As the name implies, a slack variable measures how much of a constrained resource is available, that is, how much “slack” of the resource is available. For example, recall the resource constraint used in Examples 15.1 and 15.2,

7x1 1 11x2 # 77

We can defi ne a slack variable S1 as the amount of raw gas that is not used for a particular production level (x1, x2). If this quantity is added to the left side of the constraint, it makes the relationship exact,

7x1 1 11x2 1 S1 5 77

Now recognize what the slack variable tells us. If it is positive, it means that we have some “slack” for this constraint. That is, we have some surplus resource that is not being fully utilized. If it is negative, it tells us that we have exceeded the constraint. Finally, if it is zero, we exactly meet the constraint. That is, we have used up all the allowable resource. Since this is exactly the condition where constraint lines intersect, the slack variable provides a means to detect extreme points. A different slack variable is developed for each constraint equation, resulting in what is called the fully augmented version,

Maximize Z 5 150x1 1 175x2

subject to

(15.4a)

(15.4b)

(15.4c)

(15.4d)

x1, x2, S1, S2, S3, S4 $ 0

Notice how we have set up the four equality equations so that the unknowns are aligned in columns. We did this to underscore that we are now dealing with a system of linear algebraic equations (recall Part Three). In the following section, we will show how these equations can be used to determine extreme points algebraically.

Algebraic Solution. In contrast to Part Three, where we had n equations with n un- knowns, our example system [Eqs. (15.4)] is underspecifi ed or underdetermined, that is, it has more unknowns than equations. In general terms, there are n structural variables (that is, the original unknowns), m surplus or slack variables (one per constraint), and n 1 m total variables (structural plus surplus). For the gas production problem we have 2 structural variables, 4 slack variables, and 6 total variables. Thus, the problem involves solving 4 equations with 6 unknowns.

7x1 1 11x2 1 S1 10x1 1 8x2     1 S2 x1                1 S3        x2            1 S4

5 77

5 80

5 9

5 6

cha9792x_ch15_390-415.indd Page 396 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 396 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.1 LINEAR PROGRAMMING 397

The difference between the number of unknowns and the number of equations (equal to 2 for our problem) is directly related to how we can distinguish a feasible extreme point. Specifi cally, every feasible point has 2 variables out of 6 equal to zero. For ex- ample, the fi ve corner points of the area ABCDE have the following zero values:

Extreme Point Zero Variables

A x1, x2 B x2, S2 C S1, S2 D S1, S4

E x1, S4

This observation leads to the conclusion that the extreme points can be determined from the standard form by setting two of the variables equal to zero. In our example, this reduces the problem to a solvable form of 4 equations with 4 unknowns. For example, for point E, setting x1 5 S4 5 0 reduces the standard form to

11x2 1 S1 8x2      1 S2              1 S3  x2

5 77

5 80

5 9

5 6

which can be solved for x2 5 6, S1 5 11, S2 5 32, and S3 5 9. Together with x1 5 S4 5 0, these values defi ne point E. To generalize, a basic solution for m linear equations with n unknowns is devel- oped by setting n 2 m variables to zero, and solving the m equations for the m remain- ing unknowns. The zero variables are formally referred to as nonbasic variables, whereas the remaining m variables are called basic variables. If all the basic variables are nonnegative, the result is called a basic feasible solution. The optimum will be one of these. Now a direct approach to determining the optimal solution would be to calculate all the basic solutions, determine which were feasible, and among those, which had the highest value of Z. There are two reasons why this is not a wise approach. First, for even moderately sized problems, the approach can involve solving a great number of equations. For m equations with n unknowns, this results in solving

Cnm 5 n!

m!(n 2 m)!

simultaneous equations. For example, if there are 10 equations (m 5 10) with 16 un- knowns (n 5 16), you would have 8008 [5 16!y(10! 6!)] 10 3 10 systems of equations to solve! Second, a signifi cant portion of these may be infeasible. For example, in the present problem, out of C46 5 15 extreme points, only 5 are feasible. Clearly, if we could avoid solving all these unnecessary systems, a more effi cient algorithm would be developed. Such an approach is described next.

cha9792x_ch15_390-415.indd Page 397 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 397 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

398 CONSTRAINED OPTIMIZATION

Simplex Method Implementation. The simplex method avoids ineffi ciencies outlined in the previous section. It does this by starting with a basic feasible solution. Then it moves through a sequence of other basic feasible solutions that successively improve the value of the objective function. Eventually, the optimal value is reached and the method is terminated. We will illustrate the approach using the gas-processing problem from Examples 15.1 and 15.2. The fi rst step is to start at a basic feasible solution (that is, at an extreme corner point of the feasible space). For cases like ours, an obvious starting point would be point A; that is, x1 5 x2 5 0. The original 6 equations with 4 unknowns become

S1 5 77

S2 5 80

S3 5 9

S4 5 6

Thus, the starting values for the basic variables are given automatically as being equal to the right-hand sides of the constraints. Before proceeding to the next step, the beginning information can now be sum- marized in a convenient tabular format called a tableau. As shown below, the tableau provides a concise summary of the key information constituting the linear programming problem.

Basic Z x1 x2 S1 S2 S3 S4 Solution Intercept

Z 1 2150 2175 0 0 0 0 0 S1 0 7 11 1 0 0 0 77 11 S2 0 10 8 0 1 0 0 80 8 S3 0 1 0 0 0 1 0 9 9 S4 0 0 1 0 0 0 1 6 `

Notice that for the purposes of the tableau, the objective function is expressed as

Z 2 150x1 2 175x2 2 0S1 2 0S2 2 0S3 2 0S4 5 0 (15.5)

The next step involves moving to a new basic feasible solution that leads to an improvement of the objective function. This is accomplished by increasing a current nonbasic variable (at this point, x1 or x2) above zero so that Z increases. Recall that, for the present example, extreme points must have 2 zero values. Therefore, one of the cur- rent basic variables (S1, S2, S3, or S4) must also be set to zero. To summarize this important step: one of the current nonbasic variables must be made basic (nonzero). This variable is called the entering variable. In the process, one of the current basic variables is made nonbasic (zero). This variable is called the leaving variable. Now, let us develop a mathematical approach for choosing the entering and leav- ing variables. Because of the convention by which the objective function is written [(Eq. (15.5)], the entering variable can be any variable in the objective function having a negative coeffi cient (because this will make Z bigger). The variable with the largest negative value is conventionally chosen because it usually leads to the largest increase

cha9792x_ch15_390-415.indd Page 398 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 398 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.1 LINEAR PROGRAMMING 399

in Z. For our case, x2 would be the entering variable since its coeffi cient, 2175, is more negative than the coeffi cient of x1, 2150. At this point the graphical solution can be consulted for insight. As in Fig. 15.3, we start at the initial point A. Based on its coeffi cient, x2 should be chosen to enter. However, to keep the present example brief, we choose x1 since we can see from the graph that this will bring us to the maximum quicker. Next, we must choose the leaving variable from among the current basic variables— S1, S2, S3, or S4. Graphically, we can see that there are two possibilities. Moving to point B will drive S2 to zero, whereas moving to point F will drive S1 to zero. However, the graph also makes it clear that F is not possible because it lies outside the feasible solu- tion space. Thus, we decide to move from A to B. How is the same result detected mathematically? One way is to calculate the values at which the constraint lines intersect the axis or line corresponding to the entering variable (in our case, the x1 axis). We can calculate this value as the ratio of the right- hand side of the constraint (the “Solution” column of the tableau) to the corresponding coeffi cient of x1. For example, for the fi rst constraints slack variable S1, the result is

Intercept 5 77

7 5 11

The remaining intercepts can be calculated and listed as the last column of the tableau. Because 8 is the smallest positive intercept, it means that the second constraint line will be reached fi rst as x1 is increased. Hence, S2 should be the leaving variable.

FIGURE 15.3 Graphical depiction of how the simplex method successively moves through feasible basic solu- tions to arrive at the optimum in an effi cient manner.

0

8

4

4 x1

4

1

x2

8

2

A

F

B

C

DE

3

cha9792x_ch15_390-415.indd Page 399 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 399 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

400 CONSTRAINED OPTIMIZATION

At this point, we have moved to point B (x2 5 S2 5 0), and the new basic solution becomes

7x1 1 S1      5 77

10x1        5 80

x1    1 S3      5 9

S4 5 6

The solution of this system of equations effectively defi nes the values of the basic vari- ables at point B: x1 5 8, S1 5 21, S3 5 1, and S4 5 6. The tableau can be used to make the same calculation by employing the Gauss- Jordan method. Recall that the basic strategy behind Gauss-Jordan involved converting the pivot element to 1 and then eliminating the coeffi cients in the same column above and below the pivot element (recall Sec. 9.7). For this example, the pivot row is S2 (the leaving variable) and the pivot element is 10 (the coeffi cient of the entering variable, x1). Dividing the row by 10 and replacing S2 by x1 gives

Basic Z x1 x2 S1 S2 S3 S4 Solution Intercept

Z 1 2150 2175 0 0 0 0 0 S1 0 7 11 1 0 0 0 77 x1 0 1 0.8 0 0.1 0 0 8 S3 0 1 0 0 0 1 0 9 S4 0 0 1 0 0 0 1 6

Next, the x1 coeffi cients in the other rows can be eliminated. For example, for the objective function row, the pivot row is multiplied by 2150 and the result subtracted from the fi rst row to give

Z x1 x2 S1 S2 S3 S4 Solution

1 2150 2175 0 0 0 0 0 20 2(2150) 2(2120) 20 2(215) 0 0 2(21200)

1 0 255 0 15 0 0 1200

Similar operations can be performed on the remaining rows to give the new tableau,

Basic Z x1 x2 S1 S2 S3 S4 Solution Intercept

Z 1 0 255 0 15 0 0 1200 S1 0 0 5.4 1 20.7 0 0 21 3.889 x1 0 1 0.8 0 0.1 0 0 8 10 S3 0 0 20.8 0 20.1 1 0 1 21.25 S4 0 0 1 0 0 0 1 6 6

cha9792x_ch15_390-415.indd Page 400 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 400 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.2 NONLINEAR CONSTRAINED OPTIMIZATION 401

Thus, the new tableau summarizes all the information for point B. This includes the fact that the move has increased the objective function to Z 5 1200. This tableau can then be used to chart our next, and in this case fi nal, step. Only one more variable, x2, has a negative value in the objective function, and it is therefore chosen as the entering variable. According to the intercept values (now calculated as the solution column over the coeffi cients in the x2 column), the fi rst constraint has the small- est positive value, and therefore, S1 is selected as the leaving variable. Thus, the simplex method moves us from points B to C in Fig. 15.3. Finally, the Gauss-Jordan elimination can be implemented to solve the simultaneous equations. The result is the fi nal tableau,

Basic Z x1 x2 S1 S2 S3 S4 Solution

Z 1 0 0 10.1852 7.8704 0 0 1413.889 x2 0 0 1 0.1852 20.1296 0 0 3.889 x1 0 1 0 20.1481 0.2037 0 0 4.889 S3 0 0 0 0.1481 20.2037 1 0 4.111 S4 0 0 0 20.1852 0.1296 0 1 2.111

We know that the result is fi nal because there are no negative coeffi cients remaining in the objective function row. The fi nal solution is tabulated as x1 5 3.889 and x2 5 4.889, which give a maximum objective function of Z 5 1413.889. Further, because S3 and S4 are still in the basis, we know that the solution is limited by the fi rst and second constraints.

15.2 NONLINEAR CONSTRAINED OPTIMIZATION

There are a number of approaches for handling nonlinear optimization problems in the presence of constraints. These can generally be divided into indirect and direct ap- proaches (Rao, 1996). A typical indirect approach uses so-called penalty functions. These involve placing additional expressions to make the objective function less optimal as the solution approaches a constraint. Thus, the solution will be discouraged from violating constraints. Although such methods can be useful in some problems, they can become arduous when the problem involves many constraints. The generalized reduced gradient (GRG) search method is one of the more popular of the direct methods (for details, see Fylstra et al., 1998; Lasdon et al., 1978; Lasdon and Smith, 1992). It is, in fact, the nonlinear method used within the Excel Solver. It fi rst “reduces” the problem to an unconstrained optimization problem. It does this by solving a set of nonlinear equations for the basic variables in terms of the nonbasic variables. Then, the unconstrained problem is solved using approaches similar to those described in Chap. 14. First, a search direction is chosen along which an improvement in the objective function is sought. The default choice is a quasi-Newton approach (BFGS) that, as described in Chap. 14, requires storage of an approximation of the Hessian matrix. This approach performs very well for most cases. The conjugate gradient approach is also available in Excel as an alternative for large problems. The Excel Solver has the nice feature that it automatically switches to the conjugate gradient method, depending on available storage. Once the search direction is established, a one-dimensional search is carried out along that direction using a variable step-size approach.

cha9792x_ch15_390-415.indd Page 401 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 401 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

402 CONSTRAINED OPTIMIZATION S O F T W A R E 15.3 OPTIMIZATION WITH SOFTWARE PACKAGES

Software packages have great capabilities for optimization. In this section, we will give you an introduction to some of the more useful ones.

15.3.1 Excel for Linear Programming

There are a variety of software packages expressly designed to implement linear program- ming. However, because of its broad availability, we will focus on the Excel spreadsheet. This involves using the Solver option previously employed in Chap. 7 for root location. The manner in which Solver is used for linear programming is similar to our previ- ous applications in that these data are entered into spreadsheet cells. The basic strategy is to arrive at a single cell that is to be optimized as a function of variations of other cells on the spreadsheet. The following example illustrates how this can be done for the gas-processing problem.

EXAMPLE 15.3 Using Excel’s Solver for a Linear Programming Problem

Problem Statement. Use Excel to solve the gas-processing problem we have been examining in this chapter.

Solution. An Excel worksheet set up to calculate the pertinent values in the gas- processing problem is shown in Fig. 15.4. The unshaded cells are those containing numeric and labeling data. The shaded cells involve quantities that are calculated based on other cells. Recognize that the cell to be maximized is D12, which contains the total profi t. The cells to be varied are B4:C4, which hold the amounts of regular and premium gas produced.

FIGURE 15.4 Excel spreadsheet set up to use the Solver for linear programming.

cha9792x_ch15_390-415.indd Page 402 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 402 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.3 OPTIMIZATION WITH SOFTWARE PACKAGES 403

Once the spreadsheet is created, Solver is chosen from the Data tab (recall Sec. 7.7.1). At this point a dialogue box will be displayed, querying you for pertinent information. The pertinent cells of the Solver dialogue box are fi lled out as

The constraints must be added one by one by selecting the “Add” button. This will open up a dialogue box that looks like

As shown, the constraint that the total raw gas (cell D6) must be less than or equal to the available supply (E6) can be added as shown. After adding each constraint, the “Add” button can be selected. When all four constraints have been entered, the OK but- ton is selected to return to the Solver dialogue box. Now, before execution, the Solver options button should be selected and the box la- beled “Assume linear model” should be checked off. This will make Excel employ a ver- sion of the simplex algorithm (rather than the more general nonlinear solver it usually uses) that will speed up your application. After selecting this option, return to the Solver menu. When the OK button is se- lected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the correct solution (Fig. 15.5)

cha9792x_ch15_390-415.indd Page 403 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 403 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

404 CONSTRAINED OPTIMIZATION S O F T W A R E

Beyond obtaining the solution, the Solver also provides some useful summary reports. We will explore these in the engineering application described in Sec. 16.2.

15.3.2 Excel for Nonlinear Optimization

The manner in which Solver is used for nonlinear optimization is similar to our previous applications in that these data are entered into spreadsheet cells. Once again, the basic strategy is to arrive at a single cell that is to be optimized as a function of variations of other cells on the spreadsheet. The following example illustrates how this can be done for the parachutist problem we set up in the introduction to this part of the book (recall Example PT4.1).

EXAMPLE 15.4 Using Excel‘s Solver for Nonlinear Constrained Optimization

Problem Statement. Recall from Example PT4.1 that we developed a nonlinear con- strained optimization to minimize the cost for a parachute drop into a refugee camp. Parameters for this problem are

Parameter Symbol Value Unit

Total mass Mt 2000 kg Acceleration of gravity g 9.8 m/s2

Cost coeffi cient (constant) c0 200 $ Cost coeffi cient (length) c1 56 $/m Cost coeffi cient (area) c2 0.1 $/m2

Critical impact velocity vc 20 m/s Area effect on drag kc 3 kg/(s ? m2) Initial drop height z0 500 m

FIGURE 15.5 Excel spreadsheet showing solution to linear programming problem.

cha9792x_ch15_390-415.indd Page 404 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 404 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.3 OPTIMIZATION WITH SOFTWARE PACKAGES 405

Substituting these values into Eqs. (PT4.11) through (PT4.19) gives

Minimize C 5 n(200 1 56/ 1 0.1A2)

subject to

y # 20

n $ 1

where n is an integer and all other variables are real. In addition, the following quantities are defi ned as

A 5 2pr2

/ 5 12r c 5 3A

m 5 Mt n

(E15.4.1)

t 5 root c500 2 9.8m c

t 1 9.8m2

c2 (1 2 e2(cym)t)d (E15.4.2)

y 5 9.8m

c (1 2 e2(cym)t)

Use Excel to solve this problem for the design variables r and n that minimize cost C.

Solution. Before implementation of this problem on Excel, we must fi rst deal with the problem of determining the root in the above formulation [Eq. (E15.4.2)]. One method might be to develop a macro to implement a root-location method such as bisection or the secant method. (Note that we will illustrate how this is done in the next chapter in Sec. 16.3.) For the time being, an easier approach is possible by developing the following fi xed- point iteration solution to Eq. (E15.4.2),

ti11 5 c500 1 9.8m 2

c2 (1 2 e2(cym)ti)d c

9.8m (E15.4.3)

Thus, t can be adjusted until Eq. (E15.4.3) is satisfi ed. It can be shown that for the range of parameters used in the present problem, this formula always converges. Now, how can this equation be solved on a spreadsheet? As shown below, two cells can be set up to hold a value for t and for the right-hand side of Eq. (E15.4.3) [that is, f(t)].

cha9792x_ch15_390-415.indd Page 405 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 405 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

406 CONSTRAINED OPTIMIZATION S O F T W A R E

You can type Eq. (E15.4.3) into cell B21 so that it gets its time value from cell B20 and the other parameter values from cells elsewhere on the sheet (see below for how we set up the whole sheet). Then go to cell B20 and point its value to cell B21. Once you enter these formulations, you will immediately get the error message: “Cannot resolve circular references” because B20 depends on B21 and vice versa. Now, go to the Tools/Options selections from the menu and select calculation. From the cal- culation dialogue box, check off “iteration” and hit “OK.” Immediately the spreadsheet will iterate these cells and the result will come out as

FIGURE 15.6 Excel spreadsheet set up for the nonlinear parachute optimization problem.

Thus, the cells will converge on the root. If you want to make it more precise, just strike the F9 key to make it iterate some more (the default is 100 iterations, which you can change if you wish). An Excel worksheet to calculate the pertinent values can then be set up as shown in Fig. 15.6. The unshaded cells are those containing numeric and labeling data. The

cha9792x_ch15_390-415.indd Page 406 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 406 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.3 OPTIMIZATION WITH SOFTWARE PACKAGES 407

shaded cells involve quantities that are calculated based on other cells. For example, the mass in B17 was computed with Eq. (E15.4.1) based on the values for Mt (B4) and n (E5). Note also that some cells are redundant. For example, cell E11 points back to cell E5. The information is repeated in cell E11 so that the structure of the constraints is evident from the sheet. Finally, recognize that the cell to be minimized is E15, which contains the total cost. The cells to be varied are E4:E5, which hold the radius and the number of parachutes. Once the spreadsheet is created, the selection Solver is chosen from the Data tab. At this point a dialogue box will be displayed, querying you for pertinent information. The pertinent cells of the Solver dialogue box would be fi lled out as

The constraints must be added one by one by selecting the “Add” button. This will open up a dialogue box that looks like

As shown, the constraint that the actual impact velocity (cell E10) must be less than or equal to the required velocity (G10) can be added as shown. After adding each con- straint, the “Add” button can be selected. Note that the down arrow allows you to choose among several types of constraints (,5, .5, 5, and integer). Thus, we can force the number of parachutes (E5) to be an integer. When all three constraints have been entered, the “OK” button is selected to return to the Solver dialogue box. After selecting this option return to the Solver menu. When the “OK” button is selected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the correct solution as in Fig. 15.7.

cha9792x_ch15_390-415.indd Page 407 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 407 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

408 CONSTRAINED OPTIMIZATION

Thus, we determine that the minimum cost of $4377.26 will occur if we break the load up into six parcels with a chute radius of 2.944 m. Beyond obtaining the solution, the Solver also provides some useful summary reports. We will explore these in the engineering application described in Sec. 16.2.

FIGURE 15.7 Excel spreadsheet showing the solution for the nonlinear parachute optimization problem.

15.3.3 MATLAB

As summarized in Table 15.1, MATLAB software has a variety of built-in functions to perform optimization. The following examples illustrates how they can be used.

TABLE 15.1 MATLAB functions to implement optimization.

Function Description

fminbnd Minimize function of one variable with bound constraints fminsearch Minimize function of several variables

cha9792x_ch15_390-415.indd Page 408 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 408 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.3 OPTIMIZATION WITH SOFTWARE PACKAGES 409

EXAMPLE 15.5 Using MATLAB for One-Dimensional Optimization

Problem Statement. Use the MATLAB fminbnd function to fi nd the maximum of

f(x) 5 2 sin x 2 x2

2

within the interval xl 5 0 and xu 5 4. Recall that in Chap. 13, we used several methods to solve this problem for x 5 1.7757 and f(x) 5 1.4276.

Solution. First, we must create an M-fi le to hold the function.

function f=fx(x) f = −(2*sin(x)−x^2/10)

Because we are interested in maximization, we enter the negative of the function. Then, we invoke the fminbnd function with

>> x=fminbnd('fx',0,4)

The result is

f = −1.7757

x = 1.4275

Note that additional arguments can be included. One useful addition is to set optimiza- tion options such as error tolerance or maximum iterations. This is done with the optimset function, which was used previously in Example 7.6 and has the general format,

optimset('param1',value1,'param2',value2,...)

where parami is a parameter specifying the type of option and valuei is the value assigned to that option. For example, if you wanted to set the tolerance at 1 31022,

optimset('TolX',le–2)

Thus, solving the present problem to a tolerance of 1 3 1022 can be generated with

>> fminbnd('fx',0,4,optimset('TolX',le–2))

with the result

f = −1.7757

ans = 1.4270

cha9792x_ch15_390-415.indd Page 409 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 409 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

410 CONSTRAINED OPTIMIZATION S O F T W A R E

A complete set of parameters can be found by invoking Help as in

>> Help optimset

MATLAB has a variety of capabilities for dealing with multidimensional functions. Recall from Chap. 13 that our visual image of a one-dimensional search was like a roller coaster. For two-dimensional cases, the image becomes that of mountains and valleys. As in the following example, MATLAB’s graphic capabilities provide a handy means to visualize such functions.

EXAMPLE 15.6 Visualizing a Two-Dimensional Function

Problem Statement. Use MATLAB’s graphical capabilities to display the following function and visually estimate its minimum in the range 22 # x1 # 0 and 0 # x2 # 3:

f(x1, x2) 5 2 1 x1 2 x2 1 2x 2 1 1 2x1x2 1 x

2 2

Solution. The following script generates contour and mesh plots of the function:

x=linspace(−2,0,40);y=linspace(0,3,40); [X,Y] = meshgrid(x,y); Z=2+X−Y+2*X.^2+2*X.*Y+Y.^2; subplot(1,2,1); cs=contour(X,Y,Z);clabel(cs); xlabel('x_1');ylabel('x_2'); title('(a) Contour plot');grid; subplot(1,2,2); cs=surfc(X,Y,Z); zmin=floor(min(Z)); zmax=ceil(max(Z)); xlabel('x_1');ylabel('x_2');zlabel('f(x_1,x_2)'); title('(b) Mesh plot');

As displayed in Fig. 15.8, both plots indicate that function has a minimum value of about f(x1, x2) 5 0 to 1 located at about x1 5 21 and x2 5 1.5. Standard MATLAB has a function fminsearch that can be used to determine the minimum of a multidimensional function. It is based on the Nelder-Mead method, which is a direct-search method that uses only function values (does not require derivatives) and handles nonsmooth objective functions. A simple expression of its syntax is

[xmin, fval] = fminsearch(function,x1,x2)

where xmin and fval are the location and value of the minimum, function is the name of the function being evaluated, and x1 and x2 are the bounds of the interval being searched.

cha9792x_ch15_390-415.indd Page 410 30/10/13 9:37 PM f-512 cha9792x_ch15_390-415.indd Page 410 30/10/13 9:37 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

15.3 OPTIMIZATION WITH SOFTWARE PACKAGES 411

FIGURE 15.8 (a) Contour and (b) mesh plots of a two-dimensional function.

x1

(x 1,

x 2 )

x1

(a) Contour plot (b) Mesh plot

x2

x 2

EXAMPLE 15.7 Using MATLAB for Multidimensional Optimization

Problem Statement. Use the MATLAB fminsearch function to fi nd the maximum for the simple function we just graphed in Example 15.6.

f(x1, x2) 5 2 1 x1 2 x2 1 2x  2 1 1 2x1x2 1 x

2 2

Employ initial guesses of x 5 20.5 and y 5 0.5.

Solution. We can invoke the fminsearch function with

>> f=@(x) 2+x(1)−x(2)+2*x(1)^2+2*x(1)*x(2)+x(2)^2; >> [x,fval]=fminsearch(f,[−0.5,0.5])

x = −1.0000 1.5000 fval = 0.7500

Just as with fminbnd, arguments can be included in order to specify additional param- eters of the optimization process. For example, the optimset function can be used to limit the maximum number of iterations

>> [x,fval]=fminsearch(f,[−0.5,0.5],optimset('MaxIter',2))

cha9792x_ch15_390-415.indd Page 411 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 411 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

412 CONSTRAINED OPTIMIZATION

with the result

Exiting: Maximum number of iterations has been exceeded − increase MaxIter option. Current function value: 1.225625 x = −0.5000 0.5250 fval = 1.2256

Thus, because we have set a very stringent limit on the iterations, the optimization ter- minates well before the maximum is reached.

15.3.4 Mathcad

Mathcad contains a numeric mode function called Find that can be used to solve up to 50 simultaneous nonlinear algebraic equations with inequality constraints. The use of this function for unconstrained applications was described in Part Two. If Find fails to locate a solution that satisfi es the equations and constraints, it returns the error message “did not fi nd solution.” However, Mathcad also contains a similar function called Minerr. This function gives solution results that minimize the errors in the constraints even when exact solutions cannot be found. This function solves equations and accommodates sev- eral constraints using the Levenberg-Marquardt method taken from the public-domain MINPACK algorithms developed and published by the Argonne National Laboratory. Let’s develop an example where Find is used to solve a system of nonlinear equa- tions with constraints. Initial guesses of x 5 21 and y 5 1 are input using the defi nition symbol as shown in Fig. 15.9. The word Given then alerts Mathcad that what follows is a system of equations. Then we can enter the equations and the inequality constraint. Note that for this application, Mathcad requires the use of a symbolic equal sign (typed as [Ctrl]5) and . to separate the left and right sides of an equation. Now the vector consisting of xval and yval is computed using Find (x,y) and the values are shown using an equal sign. A graph that displays the equations and constraints as well as the solution can be placed on the worksheet by clicking to the desired location. This places a red crosshair at that location. Then use the Insert/Graph/X-Y Plot pull-down menu to place an empty plot on the worksheet with placeholders for the expressions to be graphed and for the ranges of the x and y axes. Four variables are plotted on the y axis as shown: the top and bottom halves of the equation for the circle, the linear function, and a vertical line to represent the x . 2 constraint. In addition, the solution is included as a point. Once the graph has been created, you can use the Format/Graph/X-Y Plot pull-down menu to vary the type of graph; change the color, type, and weight of the trace of the function; and add titles, labels, and other features. The graph and the numerical values for xval and yval nicely portray the solution as the intersection of the circle and the line in the region where x . 2.

cha9792x_ch15_390-415.indd Page 412 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 412 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 413

FIGURE 15.9 Mathcad screen for a nonlinear constrained optimization problem.

PROBLEMS

15.1 A company makes two types of products, A and B. These products are produced during a 40-hr work week and then shipped out at the end of the week. They require 20 and 5 kg of raw material per kg of product, respectively, and the company has access to 9500 kg of raw material per week. Only one product can be created at a time with production times for each of 0.04 and 0.12 hr, respectively. The plant can only store 550 kg of total product per week. Finally, the company makes profi ts of $45 and $20 on each unit of A and B, respectively. Each unit of product is equivalent to a kg. (a) Set up the linear programming problem to maximize profi t. (b) Solve the linear programming problem graphically. (c) Solve the linear programming problem with the simplex method. (d) Solve the problem with a software package. (e) Evaluate which of the following options will raise profi ts the

most: increasing raw material, storage, or production time. 15.2 Suppose that for Example 15.1, the gas-processing plant decides to produce a third grade of product with the following characteristics:

Supreme

Raw gas 15 m3/tonne Production time 12 hr/tonne Storage 5 tonnes Profi t $250/tonne

In addition, suppose that a new source of raw gas has been discov- ered so that the total available is doubled to 154 m3/week. (a) Set up the linear programming problem to maximize profi t. (b) Solve the linear programming problem with the simplex method. (c) Solve the problem with a software package. (d) Evaluate which of the following options will raise profi ts the

most: increasing raw material, storage, or production time. 15.3 Consider the linear programming problem:

Maximize f (x, y) 5 1.75x 1 1.25y

cha9792x_ch15_390-415.indd Page 413 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 413 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

414 CONSTRAINED OPTIMIZATION

15.7 Consider the following constrained nonlinear optimization problem:

Minimize f (x, y) 5 (x 2 3)2 1 (y 2 3)2

subject to

x 1 2y 5 4

(a) Use a graphical approach to estimate the solution. (b) Use a software package (for example, Excel) to obtain a more

accurate estimate. 15.8 Use a software package to determine the maximum of

f (x, y) 5 2.25xy 1 1.75y 2 1.5x2 2 2y2

15.9 Use a software package to determine the maximum of

f (x, y) 5 4x 1 2y 1 x2 2 2x4 1 2xy 2 3y2

15.10 Given the following function,

f (x, y) 5 28x 1 x2 1 12y 1 4y2 2 2xy

use a software package to determine the minimum: (a) Graphically. (b) Numerically. (c) Substitute the result of (b) back into the function to determine

the minimum f(x, y). (d) Determine the Hessian and its determinant, and substitute the

result of part (b) back into the latter to verify that a minimum has been detected.

15.11 You are asked to design a covered conical pit to store 50 m3 of waste liquid. Assume excavation costs at $100ym3, side lining costs at $50ym2, and cover cost at 25ym2. Determine the dimen- sions of the pit that minimize cost (a) if the side slope is uncon- strained and (b) if the side slope must be less than 458. 15.12 An automobile company has two versions of the same model car for sale, a two-door coupe and the full-size four door. (a) Graphically solve how many cars of each design should be

produced to maximize profi t and what that profi t is. (b) Solve the same problem with Excel.

Two Door Four Door Availability

Profi t $13,500/car $15,000/car Production time 15 h/car 20 h/car 8000 h/year Storage 400 cars 350 cars Consumer demand 700/car 500/car 240,000 cars

15.13 Og is the leader of the surprisingly mathematically ad- vanced, though technologically run-of-the-mill, Calm Waters cave- man tribe. He must decide on the number of stone clubs and stone axes to be produced for the upcoming battle against the neighboring

subject to

1.2x 1 2.25y # 14

x 1 1.1y # 8

2.5x 1 y # 9

x $ 0

y $ 0

Obtain the solution: (a) Graphically. (b) Using the simplex method. (c) Using an appropriate software package (for example, Excel,

MATLAB, or Mathcad). 15.4 Consider the linear programming problem:

Maximize f (x, y) 5 6x 1 8y

subject to

5x 1 2y # 40

6x 1 6y # 60

2x 1 4y # 32

x $ 0

y $ 0

Obtain the solution: (a) Graphically. (b) Using the simplex method. (c) Using an appropriate software package (for example, Excel). 15.5 Use a software package (for example, Excel, MATLAB, Mathcad) to solve the following constrained nonlinear optimization problem:

Maximize f (x, y) 5 1.2x 1 2y 2 y3

subject to

2x 1 y # 2

x $ 0

y $ 0

15.6 Use a software package (for example, Excel, MATLAB, Mathcad) to solve the following constrained nonlinear optimization problem:

Maximize f (x, y) 5 15x 1 15y

subject to

x2 1 y2 # 1

x 1 2y # 2.1

x $ 0

y $ 0

cha9792x_ch15_390-415.indd Page 414 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 414 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 415

• Check whether the guesses bracket a maximum. If not, the func- tion should not implement the algorithm, but should return an error message.

• Iterate until the relative error falls below a stopping criterion or exceeds a maximum number of iterations.

• Return both the optimal x and f(x). • Use a bracketing approach (as in Example 13.2) to replace old

values with new values.

15.17 The length of the longest ladder that can negotiate the corner depicted in Fig. P15.17 can be determined by computing the value of u that minimizes the following function:

L(u) 5 w1

sin u 1

w2 sin(p 2 a 2 u)

For the case where w1 5 w2 5 2 m, use a numerical method (in- cluding software) to develop a plot of L versus a range of a’s from 458 to 1358.

Peaceful Sunset tribe. Experience has taught him that each club is good for, on the average, 0.45 kills and 0.65 maims, while each axe produces 0.70 kills and 0.35 maims. Production of a club requires 5.1 lb of stone and 2.1 man-hours of labor while an axe requires 3.2 lb of stone and 4.3 man-hours of labor. Og’s tribe has 240 lb of stone available for weapons production, and a total of 200 man-hours of labor available before the expected time of this battle (that Og is sure will end war for all time). Og values a kill as worth two maims in quantifying the damage infl icted on the enemy, and he wishes to produce that mix of weapons that will maximize damage. (a) Formulate this as a linear programming problem. Make sure to

defi ne your decision variables. (b) Represent this problem graphically, making sure to identify all

the feasible corner points and the infeasible corner points. (c) Solve the problem graphically. (d) Solve the problem using the computer. 15.14 Develop an M-fi le that is expressly designed to locate a maximum with the golden-section search algorithm. In other words, set it up so that it directly fi nds the maximum rather than fi nding the minimum of 2f(x). Test your program with the same problem as Example 13.1. The function should have the following features:

• Iterate until the relative error falls below a stopping criterion or exceeds a maximum number of iterations.

• Return both the optimal x and f(x).

15.15 Develop an M-fi le to locate a minimum with the golden- section search. Rather than using the standard stopping criteria (as in Fig. 13.5), determine the number of iterations needed to attain a desired tolerance. 15.16 Develop an M-fi le to implement parabolic interpolation to locate a minimum. Test your program with the same problem as Example 13.2. The function should have the following features:

• Base it on two initial guesses, and have the program generate the third initial value at the midpoint of the interval.

FIGURE P15.17 A ladder negotiating a corner formed by two hallways.

w2

a

q w1

L

cha9792x_ch15_390-415.indd Page 415 30/10/13 8:18 PM f-512 cha9792x_ch15_390-415.indd Page 415 30/10/13 8:18 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16 C H A P T E R 16

416

Case Studies: Optimization

The purpose of this chapter is to use the numerical procedures discussed in Chaps. 13 through 15 to solve actual engineering problems involving optimization. These prob- lems are important because engineers are often called upon to come up with the “best” solution to a problem. Because many of these cases involve complex systems and interactions, numerical methods and computers are often a necessity for developing optimal solutions. The following applications are typical of those that are routinely encountered during upper-class and graduate studies. Furthermore, they are representative of problems you will address professionally. The problems are drawn from the major discipline areas of engineering: chemical/bio, civil/environmental, electrical, and mechanical/aerospace. The fi rst application, taken from chemical/bio engineering, deals with using nonlin- ear constrained optimization to design an optimal cylindrical tank. The Excel Solver is used to develop the solution. Next, we use linear programming to assess a problem from civil/environmental en- gineering: minimizing the cost of waste treatment to meet water-quality objectives in a river. In this example, we introduce the notion of shadow prices and their use in assess- ing the sensitivity of a linear programming solution. The third application, taken from electrical engineering, involves maximizing the power across a potentiometer in an electric circuit. The solution involves one-dimensional unconstrained optimization. Aside from solving the problem, we illustrate how the Visual Basic macro language allows access to the golden-section search algorithm within the context of the Excel environment. Finally, the fourth application, taken from mechanical/aerospace engineering, involves determining the equilibrium position of a multi-spring system based on the minimum potential energy.

16.1 LEAST-COST DESIGN OF A TANK (CHEMICAL/BIO ENGINEERING)

Background. Chemical engineers (as well as other specialists such as mechanical and civil engineers) often encounter the general problem of designing containers to transport liquids and gases. Suppose that you are asked to determine the dimensions of a small

cha9792x_ch16_416-437.indd Page 416 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 416 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.1 LEAST-COST DESIGN OF A TANK 417

cylindrical tank to transport toxic waste that is to be mounted on the back of a pickup truck. Your overall objective will be to minimize the cost of the tank. However, aside from cost, you must ensure that it holds the required amount of liquid and that it does not exceed the dimensions of the truck’s bed. Note that because the tank will be carrying toxic waste, the tank thickness is specifi ed by regulations. A schematic of the tank and bed are shown in Fig. 16.1. As can be seen, the tank consists of a cylinder with two plates welded on each end. The cost of the tank involves two components: (1) material expense, which is based on weight, and (2) welding expense based on length of weld. Note that the latter involves welding both the interior and the exterior seams where the plates connect with the cylinder. The data needed for the problem are summarized in Table 16.1.

Solution. The objective here is to construct a tank for a minimum cost. The cost is related to the design variables (length and diameter) as they effect the mass of the tank and the welding lengths. Further, the problem is constrained because the tank must (1) fi t within the truck bed and (2) carry the required volume of material.

Lmax Dmax

t L

D

t

FIGURE 16.1 Parameters for determining the optimal dimensions of a cylindrical tank.

TABLE 16.1 Parameters for determining the optimal dimensions of a cylindrical tank used to transport toxic wastes.

Parameter Symbol Value Units

Required volume Vo 0.8 m3

Thickness t 3 cm Density r 8000 kg/m3

Bed length Lmax 2 m Bed width Dmax 1 m Material cost cm 4.5 $/kg Welding cost cw 20 $/m

cha9792x_ch16_416-437.indd Page 417 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 417 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

418 CASE STUDIES: OPTIMIZATION

The cost consists of tank material and welding costs. Therefore, the objective function can be formulated as minimizing

C 5 cm m 1 cw /w (16.1)

where C 5 cost ($), m 5 mass (kg), /w 5 weld length (m), and cm and cw 5 cost factors for mass ($/kg) and weld length ($/m), respectively. Next, we will formulate how the mass and weld lengths are related to the dimensions of the drum. First, the mass can be calculated as the volume of material times its density. The volume of the material used to create the side walls (that is, the cylinder) can be computed as

Vcylinder 5 Lp c aD2 1 tb 2

2 aD 2 b2 d

For each circular end plate, it is

Vplate 5 p aD2 1 tb 2

t

Thus, the mass is computed by

m 5 r eLp c aD 2

1 tb2 2 aD 2 b2 d 1 2p aD

2 1 tb2t f (16.2)

where r 5 density (kg/m3). The weld length for attaching each plate is equal to the cylinder’s inside and outside circumference. For the two plates, the total weld length would be

/w 5 2 c2p aD2 1 tb 1 2p D

2 d 5 4p(D 1 t) (16.3)

Given values for D and L (remember, thickness t is fi xed by regulations), Eqs. (16.1) through (16.3) provide a means to compute cost. Also recognize that when Eqs. (16.2) and (16.3) are substituted into Eq. (16.1), the resulting objective function is nonlinear in the unknowns. Next, we can formulate the constraints. First, we must compute how much volume can be held within the fi nished tank,

V 5 pD2

4 L

This value must be equal to the desired volume. Thus, one constraint is

pD2L

4 5 Vo

where Vo is the desired volume (m 3).

The remaining constraints deal with ensuring that the tank will fi t within the dimen- sions of the truck bed,

L # Lmax D # Dmax

cha9792x_ch16_416-437.indd Page 418 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 418 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.1 LEAST-COST DESIGN OF A TANK 419

The problem is now specifi ed. Substituting the values from Table 16.1, it can be summarized as

Maximize C 5 4.5m 1 20/w

subject to

pD2L

4 5 0.8

L # 2

D # 1

where

m 5 8000 eLp c aD 2

1 0.03b2 2 aD 2 b2 d 1 2p aD

2 1 0.03b20.03 f

and

/w 5 4p(D 1 0.03)

The problem can now be solved in a number of ways. However, the simplest approach for a problem of this magnitude is to use a tool like the Excel Solver. The spreadsheet to accomplish this is shown in Fig. 16.2. For the case shown, we enter the upper limits for D and L. For this case, the volume is more than required (1.57 . 0.8).

FIGURE 16.2 Excel spreadsheet set up to evaluate the cost of a tank subject to a volume requirement and size constraints.

cha9792x_ch16_416-437.indd Page 419 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 419 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

420 CASE STUDIES: OPTIMIZATION

Once the spreadsheet is created, the selection Solver is chosen from the Data tab. At this point a dialogue box will be displayed, querying you for pertinent information. The pertinent cells of the Solver dialogue box would be fi lled out as

When the OK button is selected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the correct solution, which is shown in Fig. 16.3. Notice that the optimal diameter is nudging up against the constraint of 1 m. Thus, if the required capacity of the tank were increased, we would run up against this constraint and the problem would reduce to a one-dimensional search for length.

FIGURE 16.3 Results of minimization. The price is reduced from $9154 to $5723 because of the smaller volume using dimensions of D 5 0.98 m and L 5 1.05 m.

cha9792x_ch16_416-437.indd Page 420 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 420 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.2 LEAST-COST TREATMENT OF WASTEWATER 421

16.2 LEAST-COST TREATMENT OF WASTEWATER (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. Wastewater discharges from big cities are often a major cause of river pollution. Figure 16.4 illustrates the type of system that an environmental engineer might confront. Several cities are located on a river and its tributary. Each generates pollution at a loading rate P that has units of milligrams per day (mg/d). The pollution loading is subject to waste treatment that results in a fractional removal x. Thus, the amount discharged to the river is the excess not removed by treatment,

Wi 5 (1 2 xi)Pi (16.4)

where Wi 5 waste discharge from the ith city. When the waste discharge enters the stream, it mixes with pollution from upstream sources. If complete mixing is assumed at the discharge point, the resulting concentration at the discharge point can be calculated by a simple mass balance,

ci 5 Wi 1 Qu cu

Qi (16.5)

where Qu 5 fl ow (L/d), cu 5 concentration (mg/L) in the river immediately upstream of the discharge, and Qi 5 fl ow downstream of the discharge point (L/d). After the concentration at the mixing point is established, chemical and biological decomposition processes can remove some of the pollution as it fl ows downstream. For the present case, we will assume that this removal can be represented by a simple frac- tional reduction factor R. Assuming that the headwaters (that is, the river above cities 1 and 2) are pollution- free, the concentrations at the four nodes can be computed as

c1 5 (1 2 x1)P1

Q13

c2 5 (1 2 x2)P2

Q23

c3 5 R13 Q13 c1 1 R23 Q23 c2 1 (1 2 x3)P3

Q34 (16.6)

c4 5 R34 Q34 c3 1 (1 2 x4)P4

Q45

FIGURE 16.4 Four wastewater treatment plants discharging pollution to a river system. The river segments between the cities are labeled with circled numbers.

4

P1

3

2

P4

P2

P3

W1

W2

W3 W434

23

13

45

WWTP2

1

WWTP1

WWTP4

WWTP3

cha9792x_ch16_416-437.indd Page 421 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 421 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

422 CASE STUDIES: OPTIMIZATION

Next, it is recognized that the waste treatment costs a different amount, di ($1000/mg removed), at each of the facilities. Thus, the total cost of treatment (on a daily basis) can be calculated as

Z 5 d1 P1x1 1 d2 P2x2 1 d3 P3x3 1 d4 P4x4 (16.7)

where Z is total daily cost of treatment ($1000/d). The fi nal piece in the “decision puzzle” involves environmental regulations. To pro- tect the benefi cial uses of the river (for example, boating, fi sheries, bathing), regulations say that the river concentration must not exceed a water-quality standard of cs. Parameters for the river system in Fig. 16.4 are summarized in Table 16.2. Notice that there is a difference in treatment cost between the upstream (1 and 2) and the down- stream cities (3 and 4) because of the outmoded nature of the downstream plants. The concentration can be calculated with Eq. (16.6) and the result listed in the shaded column for the case where no waste treatment is implemented (that is, all the x’s 5 0). Notice that the standard of 20 mg/L is being violated at all mixing points. Use linear programming to determine the treatment levels that meet the water-quality standards for the minimum cost. Also, evaluate the impact of making the standard more stringent below city 3. That is, redo the exercise, but with the standards for segments 3–4 and 4–5 lowered to 10 mg/L.

Solution. All the factors outlined above can be combined into the following linear programming problem:

Minimize Z 5 d1P1x1 1 d2P2x2 1 d3P3x3 1 d4P4x4 (16.8)

subject to the following constraints

(1 2 x1)P1 Q13

# cs1

(1 2 x2)P2 Q23

# cs2

R13Q13c1 1 R23Q23c2 1 (1 2 x3)P3 Q34

# cs3 (16.9)

R34Q34c3 1 (1 2 x4)P4 Q45

# cs4

0 # x1, x2, x3, x4 # 1 (16.10)

TABLE 16.2 Parameters for four wastewater treatment plants discharging pollution to a river system, along with the resulting concentration (ci) for zero treatment. Flow, removal, and standards for the river segments are also listed.

City Pi (mg/d) di ($1026/mg) ci (mg/L) Segment Q (L/d) R cs (mg/L)

1 1.00 3 109 2 100 1–3 1.00 3 107 0.5 20 2 2.00 3 109 2 40 2–3 5.00 3 107 0.35 20 3 4.00 3 109 4 47.3 3–4 1.10 3 108 0.6 20 4 2.50 3 109 4 22.5 4–5 2.50 3 108 20

cha9792x_ch16_416-437.indd Page 422 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 422 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.2 LEAST-COST TREATMENT OF WASTEWATER 423

Thus, the objective function is to minimize treatment cost [Eq. (16.8)] subject to the constraint that water-quality standards must be met for all parts of the system [Eq. (16.9)]. In addition, treatment cannot be negative or greater than 100% removal [Eq. (16.10)]. The problem can be solved using a variety of packages. For the present application, we use the Excel spreadsheet. As seen in Fig. 16.5, these data along with the concentra- tion calculations can be set up nicely in the spreadsheet cells. Once the spreadsheet is created, the selection Solver is chosen from the Data tab. At this point a dialogue box will be displayed, querying you for pertinent information. The pertinent cells of the Solver dialogue box would be fi lled out as

FIGURE 16.5 Excel spreadsheet set up to evaluate the cost of waste treatment on a regulated river system. Column F contains the calculation of concentration according to Eq. (16.6). Cells F4 and H4 are highlighted to show the formulas used to calculate c1 and treatment cost for city 1. In addition, highlighted cell H9 shows the formula (Eq. 16.8) for total cost that is to be minimized.

Notice that not all the constraints are shown, because the dialogue box displays only six constraints at a time.

cha9792x_ch16_416-437.indd Page 423 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 423 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

424 CASE STUDIES: OPTIMIZATION

When the OK button is selected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the correct solution, which is shown in Fig. 16.6. Before accepting the solution (by selecting the OK button on the Solver Reports box), notice that three reports can be generated: Answer, Sensitivity, and Limits. Select the Sensitivity Report and then hit the OK button to accept the solution. The Solver will automatically generate a Sensitivity Report, as in Fig. 16.7. Now let us examine the solution (Fig. 16.6). Notice that the standard will be met at all the mixing points. In fact, the concentration at city 4 will actually be less than the standard (16.28 mg/L), even though no treatment would be required for city 4. As a fi nal exercise, we can lower the standards for reaches 3–4 and 4–5 to 10 mg/L. Before doing this, we can examine the Sensitivity Report. For the present case, the key column of Fig. 16.7 is the Lagrange Multiplier (aka the “shadow price”). The shadow price is a value that expresses the sensitivity of the objective function (in our case, cost) to a unit change of one of the constraints (water-quality standards). It therefore represents the additional cost that will be incurred by making the standards more stringent. For our example, it is revealing that the largest shadow price, 2$440yDcs3, occurs for one of the standard changes (that is, downstream from city 3) that we are contemplating. This tips us off that our modifi cation will be costly. This is confi rmed when we rerun Solver with the new standards (that is, we lower cells G6 and G7 to 10). As seen in Table 16.3, the result is that treatment cost is increased from $12,600/day to $19,640/day. In addition, reducing the standard concentrations for the lower reaches means that city 4 must begin to treat its waste and city 3 must upgrade its treatment. Notice also that the treatment of the upstream cities is unaffected.

FIGURE 16.6 Results of minimization. The water-quality standards are met at a cost of $12,600/day. Notice that despite the fact that no treatment is required for city 4, the concentration at its mixing point actually exceeds the standard.

cha9792x_ch16_416-437.indd Page 424 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 424 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.3 MAXIMUM POWER TRANSFER FOR A CIRCUIT 425

16.3 MAXIMUM POWER TRANSFER FOR A CIRCUIT (ELECTRICAL ENGINEERING)

Background. The simple resistor circuit in Fig. 16.8 contains three fi xed resistors and one adjustable resistor. Adjustable resistors are called potentiometers. The values for the parameters are V 5 80 V, R1 5 8 V, R2 5 12 V, and R3 5 10 V. (a) Find the value of the adjustable resistance Ra that maximizes the power transfer across terminals 1 and 2. (b) Perform a sensitivity analysis to determine how the maximum power and the corresponding setting of the potentiometer (Ra) varies as V is varied over the range from 45 to 105 V.

FIGURE 16.7 Sensitivity Report for spread- sheet set up to evaluate the cost of waste treatment on a regu- lated river system.

TABLE 16.3 Comparison of two scenarios involving the impact of different regulations on treatment costs.

Scenario 1: All cs 5 20 Scenario 2: Downstream cs 5 10

City x c City x c

1 0.8 20 1 0.8 20 2 0.5 20 2 0.5 20 3 0.5625 20 3 0.8375 10 4 0 15.28 4 0.264 10

Cost 5 $12,600 Cost 5 $19,640

cha9792x_ch16_416-437.indd Page 425 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 425 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

426 CASE STUDIES: OPTIMIZATION

Solution. An expression for power for the circuit can be derived from Kirchhoff’s laws as

P(Ra) 5

c VR3 Ra R1(Ra 1 R2 1 R3) 1 R3 Ra 1 R3 R2

d 2

Ra (16.11)

Substituting the parameter values gives the plot shown in Fig. 16.9. Notice that a maximum power transfer occurs at a resistance of about 16 V. We will solve this problem in two ways with the Excel spreadsheet. First, we will employ trial-and-error and the Solver option. Then, we will develop a Visual Basic macro program to perform the sensitivity analysis. (a) An Excel spreadsheet to implement Eq. (16.11) is shown in Fig. 16.10. As indi- cated, Eq. (16.11) can be entered into cell B9. Then the value of Ra (cell B8) can be varied in a trial-and-error fashion until a minimum drag was determined. For this ex- ample, the result is a power of 30.03 W and a potentiometer setting of Ra 5 16.44 V. A superior approach involves using the Solver option from the spreadsheet’s Data tab. At this point a dialogue box will be displayed, querying you for pertinent informa- tion. The pertinent cells of the Solver dialogue box would be fi lled out as

R3

1

2

V

R2R1

Ra FIGURE 16.8 A resistor circuit with an adjustable resistor, or potentiometer.

50 100

40

0 0

Ra

P(Ra)

20

Maximum power

FIGURE 16.9 A plot of power transfer across terminals 1-2 from Fig. 16.8 as a function of the potentiometer resistance Ra.

Set target cell: B9

Equal to ● max ❍ min ❍ equal to 0

By changing cells B8

cha9792x_ch16_416-437.indd Page 426 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 426 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.3 MAXIMUM POWER TRANSFER FOR A CIRCUIT 427

When the OK button is selected, a dialogue box will open with a report on the success of the operation. For the present case, the Solver obtains the same correct solution shown in Fig. 16.10. (b) Now, although the foregoing approach is excellent for a single evaluation, it is not convenient for cases where multiple optimizations would be employed. Such would be the case for the second part of this application, where we are interested in determin- ing how the maximum power varies for different voltage settings. Of course, the Solver could be invoked multiple times for different parameter values, but this would be inef- fi cient. A preferable course would involve developing a macro function to come up with the optimum. Such a function is listed in Fig. 16.11. Notice how closely it resembles the golden- section-search pseudocode previously presented in Fig. 13.5. In addition, notice that a function must also be defi ned to compute power according to Eq. (16.11). An Excel spreadsheet utilizing this macro to evaluate the sensitivity of the solution to voltage is given in Fig. 16.12. A column of values is set up that spans the range of V’s (that is, from 45 to 105 V). A function call to the macro is written in cell B9 that references the adjacent value of V (the 45 in A9). In addition, the other parameters in the function argument are also included. Notice that, whereas the reference to V is rela- tive, the references to the lower and upper guesses and the resistances are absolute (that is, including leading $). This was done so that when the formula is copied down, the absolute references stay fi xed, whereas the relative reference corresponds to the voltage in the same row. A similar strategy is used to place Eq. (16.11) in cell C9. When the formulas are copied downward, the result is as shown in Fig. 16.12. The maximum power can be plotted to visualize the impact of voltage variations. As seen in Fig. 16.13, the power increases with V. The results for the corresponding potentiometer settings (Ra) are more interesting. The spreadsheet indicates that the same setting, 16.44 V, results in maximum power. Such a result might be diffi cult to intuit based on casual inspection of Eq. (16.11).

FIGURE 16.10 Excel determination of maximum power across a potentiometer using trial-and-error.

cha9792x_ch16_416-437.indd Page 427 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 427 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

428 CASE STUDIES: OPTIMIZATION

Option Explicit

Function Golden(xlow, xhigh, R1, R2, R3, V) Dim iter As Integer, maxit As Integer, ea As Double, es As Double Dim fx As Double, xL As Double, xU As Double, d As Double, x1 as Double Dim x2 As Double, f1 As Double, f2 As Double, xopt As Double Const R As Double = (5 ^ 0.5 – 1) / 2 maxit = 50 es = 0.001 xL = xlow xU = xhigh iter = 1 d = R * (xU – xL) x1 = xL + d x2 = xU – d f1 = f(x1, R1, R2, R3, V) f2 = f(x2, R1, R2, R3, V) If f1 > f2 Then xopt = x1 fx = f1 Else xopt = x2 fx = f2 End If Do d = R * d If f1 > f2 Then xL = x2 x2 = x1 x1 = xL + d f2 = f1 f1 = f(x1, R1, R2, R3, V) Else xU = x1 x1 = x2 x2 = xU – d f1 = f2 f2 = f(x2, R1, R2, R3, V) End If iter = iter + 1 If f1 > f2 Then xopt = x1 fx =f1 Else xopt = x2 fx = f2 End If If xopt <> 0 Then ea = (1 – R) * Abs((xU – xL) / xopt) * 100 If ea <= es Or iter >= maxit Then Exit Do Loop Golden = xopt End Function

Function f(Ra, R1, R2, R3, V) f = (V * R3 * Ra / (R1 * (Ra + R2 + R3) + R3 * Ra + R3 * R2)) ^ 2 / Ra END FUNCTION

FIGURE 16.11 Excel macro written in Visual Basic to determine a maximum with the golden-section search.

cha9792x_ch16_416-437.indd Page 428 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 428 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

16.4 EQUILIBRIUM AND MINIMUM POTENTIAL ENERGY 429

FIGURE 16.13 Results of sensitivity analysis of the effect of voltage variations on maximum power.

45 105

40

60

0

P (W)

75

20 Ra (�)

V (V)

16.4 EQUILIBRIUM AND MINIMUM POTENTIAL ENERGY (MECHANICAL/AEROSPACE ENGINEERING)

Background. As in Figure 16.14a, an unloaded spring can be attached to a wall mount. When a horizontal force is applied the spring stretches. The displacement is related to the force by Hooke’s law, F 5 kx. The potential energy of the deformed state consists of the difference between the strain energy of the spring and the work done by the force,

PE(x) 5 0.5k x2 2 F x (16.12)

Equation (16.12) defi nes a parabola. Since the potential energy will be at a minimum at equilibrium, the solution for displacement can be viewed as a one-dimensional optimization

FIGURE 16.12 Excel spreadsheet to implement a sensitivity analysis of the maximum power to variations of voltage. This routine accesses the macro program for golden-section search from Fig. 16.11.

=(A9*$B$5*B9/($B$3*(B9+$B$4+$B$5)+$B$5*B9+$B$3*$B$4))^2/B9

= Golden($B$6,$B$7,$B$3,$B$4,$B$5,A9)

Call to Visual Basic macro function

Power calculation

A B C D 1 Maximum Power Transfer 2 3 R1 8 4 R2 12 5 R3 10 6 Rmin 0.1 7 Rmax 100 8 V Ra P(Ra) 9 45 16.44444 9.501689 10 60 16.44444 16.89189 11 75 16.44444 26.39358 12 90 16.44444 38.00676 13 105 16.44444 51.73142

cha9792x_ch16_416-437.indd Page 429 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 429 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

430 CASE STUDIES: OPTIMIZATION

problem. Because this equation is so easy to differentiate, we can solve for the displacement as x 5 Fyk. For example, if k 5 2 N/cm and F 5 5 N, x 5 5Ny(2 N/cm)y5 5 2.5 cm. A more interesting two-dimensional case is shown in Figure 16.15. In this system, there are two degrees of freedom in that the system can move both horizontally and vertically. In the same way that we approached the one-dimensional system, the equilib- rium deformations are the values of x1 and x2 that minimize the potential energy,

PE(x1, x2) 5 0.5ka(2x21 1 (La 2 x2)2 2 La)2

1 0.5kb(2x21 1 (Lb 1 x2)2 2 Lb)2 2 F1x1 2 F2x2 (16.13)

If the parameters are ka 5 9 N/cm, kb 5 2 N/cm, La 5 10 cm, Lb 5 10 cm, F1 5 2 N, and F2 5 4 N, solve for the displacements and the potential energy.

Background. We can use a variety of software tools to solve this problem. For example, using MATLAB, an M-fi le can be developed to hold the potential energy function,

function p=PE(x,ka,kb,La,Lb,F1,F2) PEa=0.5*ka*(sqrt(x(1)^2+(La-x(2))^2)-La)^2; PEb=0.5*kb*(sqrt(x(1)^2+(Lb+x(2))^2)-Lb)^2; W=F1*x(1)+F2*x(2); p=PEa+PEb-W;

The solution can then be obtained with the fminsearch function,

>> ka=9;kb=2;La=10;Lb=10;F1=2;F2=4; >> [x,f]=fminsearch(@PE,[—0.5,0.5],[],ka,kb,La,Lb,F1,F2) x = 4.9523 1.2769 f = -9.6422

Thus, at equilibrium, the potential energy is 29.6422 N?cm. The connecting point is located 4.9523 cm to the right and 1.2759 cm above its original position.

FIGURE 16.14 (a) An unloaded spring at- tached to a wall mount. (b) Ap- plication of a horizontal force stretches the spring where the relationship between force and displacement is described by Hooke’s law.

k (a)

(b)

x

F

FIGURE 16.15 A two-spring system: (a) unloaded, and (b) loaded.

F1

x1

x2

ka

kbLb

La

F2

(a) (b)

cha9792x_ch16_416-437.indd Page 430 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 430 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 431

PROBLEMS

Chemical/Bio Engineering 16.1 Design the optimal cylindrical container (Fig. P16.1) that is open at one end and has walls of negligible thickness. The con- tainer is to hold 0.5 m3. Design it so that the areas of its bottom and sides are minimized. 16.2 (a) Design the optimal conical container (Fig. P16.2) that has a cover and has walls of negligible thickness. The container is to hold 0.5 m3. Design it so that the areas of its top and sides are mini- mized. (b) Repeat (a) but for a conical container without a cover. 16.3 Design the optimal cylindrical tank with dished ends (Fig. P16.3). The container is to hold 0.5 m3 and has walls of negli- gible thickness. Note that the area and volume of each of the dished ends can be computed with

A 5 p(h2 1 r2)  V 5 ph(h2 1 3r2)

6 (a) Design the tank so that its surface area is minimized. Interpret

the result. (b) Repeat part (a), but add the constraint L $ 2h.

FIGURE P16.1 A cylindrical container with no lid.

h

rOpen

FIGURE P16.2 A conical container with a lid.

h

rLid

FIGURE P16.3

L

h

r

16.4 The specifi c growth rate of a yeast that produces an antibiotic is a function of the food concentration c,

g 5 2c

4 1 0.8c 1 c2 1 0.2c3

As depicted in Fig. P16.4, growth goes to zero at very low concen- trations due to food limitation. It also goes to zero at high concen- trations due to toxicity effects. Find the value of c at which growth is a maximum. 16.5 A chemical plant makes three major products on a weekly basis. Each of these products requires a certain quantity of raw chemical and different production times, and yields different profi ts. The pertinent information is in Table P16.5. Note that there is suffi cient warehouse space at the plant to store a total of 450 kg/week.

FIGURE P16.4 The specifi c growth rate of a yeast that produces an antibiotic versus the food concentration.

5 10

0.4

0 0

c (mg/L)

g (d�1) 0.2

cha9792x_ch16_416-437.indd Page 431 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 431 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

432 CASE STUDIES: OPTIMIZATION

that the initial cost of the system is a function of the conversion xA. Find the conversion that will result in the lowest cost system. C is a proportionality constant.

Cost 5 C c a 1 (1 2 xA)

2b 0.6

1 5 a 1 xA b0.6 d

16.9 In problem 16.8, only one reactor is used. If two reactors are used in series, the governing equation for the system changes. Find the conversions for both reactors (xA1 and xA2) such that the total cost of the system is minimized.

Cost 5

C c a xA1 xA2(1 2 xA1)

2b 0.6

1 a 1 2 ( xA1 xA2)

(1 2 xA2) 2b

0.6

1 5 a 1 xA2 b0.6 d

16.10 For the reaction:

2A 1 B 3 C

equilibrium can be expressed as:

K 5 [C]

[A]2[B] 5

[C]

[A0 2 2C] 2[B0 2 C]

If K 5 2 M21, the initial concentration of A (A0) can be varied. The initial concentration of B is fi xed by the process, B0 5 100. A costs $1/M and C sells for $10/M. What would be the optimum initial concentration of A to use such that the profi ts would be maximized? 16.11 A chemical plant requires 106 L/day of a solution. Three sources are available at different prices and supply rates. Each source also has a different concentration of an impurity that must be kept below a minimum level to prevent interference with the chemical. The data for the three sources are summarized in the following table. Determine the amount from each source to meet the requirements at the least cost.

Source 1 Source 2 Source 3 Required

Cost ($yL) 0.50 1.00 1.20 minimize Supply (105 Lyday) 20 10 5 $10 Concentration (mgyL) 135 100 75 #100

(a) Set up a linear programming problem to maximize profi t. (b) Solve the linear programming problem with the simplex method. (c) Solve the problem with a software package. (d) Evaluate which of the following options will raise profi ts the

most: increasing raw chemical, production time, or storage. 16.6 Recently chemical engineers have become involved in the area known as waste minimization. This involves the operation of a chemical plant so that impacts on the environment are minimized. Suppose a refi nery develops a product Z1 made from two raw materials X and Y. The production of 1 metric tonne of the product involves 1 tonne of X and 2.5 tonnes of Y and produces 1 tonne of a liquid waste W. The engineers have come up with three alternative ways to handle the waste:

• Produce a tonne of a secondary product Z2 by adding an addi- tional tonne of X to each tonne of W.

• Produce a tonne of another secondary product Z3 by adding an additional tonne of Y to each tonne of W.

• Treat the waste so that it is permissible to discharge it.

The products yield profi ts of $2500, 2$50, and $200/tonne for Z1, Z2, and Z3, respectively. Note that producing Z2 actually creates a loss. The treatment process costs $300/tonne. In addition, the com- pany has access to a limit of 7500 and 10,000 tonnes of X and Y, respectively, during the production period. Determine how much of the products and waste must be created in order to maximize profi t. 16.7 A mixture of benzene and toluene are to be separated in a fl ash tank. At what temperature should the tank be operated to get the highest purity toluene in the liquid phase (maximizing xT)? The pres- sure in the fl ash tank is 800 mm Hg. The units for Antoine’s equation are mm Hg and 8C for pressure and temperature, respectively.

xB PsatB 1 xT PsatT 5 P

log10 (PsatB) 5 6.905 2 1211

T 1 221

log10 (PsatT) 5 6.953 2 1344

T 1 219

16.8 A compound A will be converted into B in a stirred tank reactor. The product B and unreacted A are purifi ed in a separation unit. Unreacted A is recycled to the reactor. A process engineer has found

TABLE P16.5

Resource Product 1 Product 2 Product 3 Availability

Raw chemical 7 kg/kg 5 kg/kg 13 kg/kg 3000 kg Production time 0.05 hr/kg 0.1 hr/kg 0.2 hr/kg 55 hr/week Profi t $30/kg $30/kg $35/kg

cha9792x_ch16_416-437.indd Page 432 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 432 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 433

16.16 Suppose that you are asked to design a column to support a compressive load P, as shown in Fig. P16.16a. The column has a cross-section shaped as a thin-walled pipe as shown in Fig. P16.16b.

The design variables are the mean pipe diameter d and the wall thickness t. The cost of the pipe is computed by

Cost 5 f (t, d) 5 c1W 1 c2d

where c1 5 4 and c2 5 2 are cost factors and W 5 weight of the pipe,

W 5 p dt Hr

where r 5 density of the pipe material 5 0.0025 kg/cm3. The col- umn must support the load under compressive stress and not buckle. Therefore,

Actual stress (s) # maximum compressive yield stress 5 sy 5 550 kg/cm

2

Actual stress # buckling stress

16.12 You must design a triangular open channel to carry a waste stream from a chemical plant to a waste stabilization pond (Fig.  P16.12). The mean velocity increases with the hydraulic radius Rh 5 Ayp, where A is the cross-sectional area and p equals the wetted perimeter. Because the maximum fl ow rate corresponds to the maximum velocity, the optimal design amounts to minimiz- ing the wetted perimeter. Determine the dimensions to minimize the wetted perimeter for a given cross-sectional area. Are the relative dimensions universal? 16.13 As an agricultural engineer, you must design a trapezoi- dal open channel to carry irrigation water (Fig. P16.13). Deter- mine the optimal dimensions to minimize the wetted perimeter for a cross-sectional area of 100 m2. Are the relative dimensions universal? 16.14 Find the optimal dimensions for a heated cylindrical tank designed to hold 10 m3 of fl uid. The ends and sides cost $200/m2 and $100/m2, respectively. In addition, a coating is applied to the entire tank area at a cost of $50/m2.

Civil/Environmental Engineering 16.15 A fi nite-element model of a cantilever beam subject to load- ing and moments (Fig. P16.15) is given by optimizing

f(x, y) 5 5x2 2 5xy 1 2.5y2 2 x 2 1.5y

where x 5 end displacement and y 5 end moment. Find the values of x and y that minimize f(x, y).

FIGURE P16.12

w

d

��

FIGURE P16.13

w

d

��

FIGURE P16.15 A cantilever beam.

x

y

FIGURE P16.16 (a) A column supporting a compressive load P. (b) The column has a cross section shaped as a thin-walled pipe.

(a)

H

P

(b)

d

t

cha9792x_ch16_416-437.indd Page 433 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 433 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

434 CASE STUDIES: OPTIMIZATION

below the point discharge. This point is called “critical” because it represents the location where biota that depend on oxygen (like fi sh) would be the most stressed. Determine the critical travel time and concentration, given the following values:

os 5 10 mg/L  kd 5 0.1 d 21   ka 5 0.6 d

21

ks 5 0.05 d 21  Lo 5 50 mg/L   Sb 5 1 mg/L/d

16.18 The two-dimensional distribution of pollutant concentration in a channel can be described by

c(x, y) 5 7.9 1 0.13x 1 0.21y 2 0.05x2

2 0.016y2 2 0.007x y

Determine the exact location of the peak concentration given the function and the knowledge that the peak lies within the bounds 210 # x # 10 and 0 # y # 20. 16.19 The fl ow Q (m3/s) in an open channel can be predicted with the Manning equation

Q 5 1 n

Ac R 2y3 S1y2

where n 5 Manning roughness coeffi cient (a dimensionless num- ber used to parameterize the channel friction), Ac 5 cross-sectional area of the channel (m2), S 5 channel slope (dimensionless, meters drop per meter length), and R 5 hydraulic radius (m), which is re- lated to more fundamental parameters by R 5 AcyP, where P 5 wetted perimeter (m). As the name implies, the wetted perimeter is the length of the channel sides and bottom that is under water. For example, for a rectangular channel, it is defi ned as P 5 B 1 2H, where H 5 depth (m). Suppose that you are using this formula to design a lined canal (note that farmers line canals to minimize leak- age losses). (a) Given the parameters n 5 0.035, S 5 0.003, and Q 5 1 m3/s,

determine the values of B and H that minimize the wetted pe- rimeter. Note that such a calculation would minimize cost if lining costs were much larger than excavation costs.

(b) Repeat part (a), but include the cost of excavation. To do this minimize the following cost function,

C 5 c1 Ac 1 c2 P

where c1 is a cost factor for excavation 5 $100/m 2 and c2 is a

cost factor for lining $50/m. (c) Discuss the implications of your results. 16.20 A cylindrical beam carries a compression load P 5 3000 kN. To prevent the beam from buckling, this load must be less than a critical load,

Pc 5 p2EI

L2

The actual stress is given by

s 5 P

A 5

P

pdt

The buckling stress can be shown to be

sb 5 pEI

H2dt

where E 5 modulus of elasticity and I 5 second moment of the area of the cross section. Calculus can be used to show that

I 5 p

8 dt(d2 1 t2)

Finally, diameters of available pipes are between d1 and d2 and thicknesses between t1 and t2. Develop and solve this problem by determining the values of d and t that minimize the cost. Note that H 5 275 cm, P 5 2000 kg, E 5 900,000 kg/cm2, d1 5 1 cm, d2 5 10 cm, t1 5 0.1 cm, and t2 5 1 cm. 16.17 The Streeter-Phelps model can be used to compute the dissolved oxygen concentration in a river below a point discharge of sewage (Fig. P16.17),

o 5 os 2 kd Lo

kd 1 ks 2 ka (e2ka t 2 e2(kd1ks)t) 2

Sb ka

(1 2 e2ka t)

(P16.17)

where o 5 dissolved oxygen concentration (mg/L), os 5 oxygen saturation concentration (mg/L), t 5 travel time (d), Lo 5 biochem- ical oxygen demand (BOD) concentration at the mixing point (mg/L), kd 5 rate of decomposition of BOD (d

21), ks 5 rate of set- tling of BOD (d21), ka 5 reaeration rate (d

21), and Sb 5 sediment oxygen demand (mg/L/d).

As indicated in Fig. P16.17, Eq. (P16.17) produces an oxygen “sag” that reaches a critical minimum level oc some travel time tc

15 20

8

12

0 0

t (d) 5

4

10

o (mg/L) o

os

tc

oc

FIGURE P16.17 A dissolved oxygen “sag” below a point discharge of sewage into a river.

cha9792x_ch16_416-437.indd Page 434 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 434 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 435

16.24 A system consists of two power plants that must deliver loads over a transmission network. The costs of generating power at plants 1 and 2 are given by

F1 5 2p1 1 2

F2 5 10p2

where p1 and p2 5 power produced by each plant. The losses of power due to transmission L are given by

L1 5 0.2p1 1 0.1p2 L2 5 0.2p1 1 0.5p2

The total demand for power is 30 and p1 must not exceed 42. Determine the power generation needed to meet demands while minimizing cost using an optimization routine such as those found in, for example, Excel, MATLAB, or Mathcad software. 16.25 The torque transmitted to an induction motor is a function of the slip between the rotation of the stator fi eld and the rotor speed s where slip is defi ned as

s 5 n 2 nR

n

where n 5 revolutions per second of rotating stator speed and nR 5 rotor speed. Kirchhoff’s laws can be used to show that the torque (expressed in dimensionless form) and slip are related by

T 5 15(s 2 s2)

(1 2 s) (4s2 2 3s 1 4)

Figure P16.25 shows this function. Use a numerical method to determine the slip at which the maximum torque occurs. 16.26 (a) A computer equipment manufacturer produces scanners and

printers. The resources needed for producing these devices and the corresponding profi ts are

Device Capital ($/unit) Labor (hr/unit) Profi t ($/unit)

Scanner 300 20 500 Printer 400 10 400

where E 5 Young’s modulus 5 200 3 109 N/m2, I 5 pr4y4 (the area moment of inertia for a cylindrical beam of radius r), and L is the beam length. If the volume of beam V cannot exceed 0.075 m3, fi nd the largest height L that can be utilized and the correspond- ing radius. 16.21 The Splash River has a fl ow rate of 2 3 106 m3/d, of which up to 70% can be diverted into two channels where it fl ows through Splish County. These channels are used for transportation, irriga- tion, and electric power generation, with the latter two being sources of revenue. The transportation use requires a minimum di- verted fl ow rate of 0.3 3 106 m3/d for Channel 1 and 0.2 3 106 m3/d for Channel 2. For political reasons it has been decided that the absolute difference between the fl ow rates in the two channels can- not exceed 40% of the total fl ow diverted into the channels. The Splish County Water Management Board has also limited mainte- nance costs for the channel system to be no more than $1.8 3 106 per year. Annual maintenance costs are estimated based on the daily fl ow rate. Channel 1 costs per year are estimated by multiply- ing $1.1 times the m3/d of fl ow; while for Channel 2 the multiplica- tion factor is $1.4 per m3/d. Electric power production revenue is also estimated based on daily fl ow rate. For Channel 1 this is $4.0 per m3/d, while for Channel 2 it is $3.0 per m3/d. Annual revenue from irrigation is also estimated based on daily fl ow rate, but the fl ow rates must fi rst be corrected for water loss in the channels previous to delivery for irrigation. This loss is 30% in Channel 1 and 20% in Channel 2. In both channels the reve- nue is $3.2 per m3/d. Determine the fl ows in the channels that maximize profi t. 16.22 Determine the beam cross-sectional areas that result in the minimum weight for the truss we studied in Sec. 12.2 (Fig. 12.4). The critical buckling and maximum tensile strengths of compres- sion and tension members are 10 and 20 ksi, respectively. The truss is to be constructed of steel (density 5 3.5 lb/ft-in2). Note that the length of the horizontal member (2) is 50 ft. Also, recall that the stress in each member is equal to the force divided by cross-sectional area. Set up the problem as a linear program- ming problem. Obtain the solution graphically and with the Excel Solver.

Electrical Engineering 16.23 A total charge Q is uniformly distributed around a ring- shaped conductor with radius a. A charge q is located at a distance x from the center of the ring (Fig. P16.23). The force exerted on the charge by the ring is given by

F 5 1

4pe0

q Qx

(x2 1 a2)3y2

where e0 5 8.85 3 10 212 C2y(N m2), q 5 Q 5 2 3 1025 C, and a 5

0.9 m. Determine the distance x where the force is a maximum.

FIGURE P16.23

x

a

Q

q

cha9792x_ch16_416-437.indd Page 435 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 435 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

436 CASE STUDIES: OPTIMIZATION

where D 5 drag, s 5 ratio of air density between the fl ight altitude and sea level, W 5 weight, and V 5 velocity. As seen in Fig. P16.28, the two factors contributing to drag are affected differently as velocity increases. Whereas friction drag increases with velocity, the drag due to lift decreases. The combination of the two factors leads to a minimum drag. (a) If s 5 0.5 and W 5 15,000, determine the minimum drag and

the velocity at which it occurs. (b) In addition, develop a sensitivity analysis to determine how this

optimum varies in response to a range of W 5 12,000 to 18,000 with s 5 0.5.

16.29 Roller bearings are subject to fatigue failure caused by large contact loads F (Fig. P16.29).

If there are $127,000 worth of capital and 4270 hr of labor available each day, how many of each device should be pro- duced per day to maximize profi t?

(b) Repeat the problem, but now assume that the profi t for each printer sold Pp depends on the number of printers produced Xp, as in

Pp 5 400 2 Xp

16.27 A manufacturer provides specialized microchips. During the next 3 months, its sales, costs, and available time are

Month 1 Month 2 Month 3

Chips required 1000 2500 2200 Cost regular time ($/chip) 100 100 120 Cost overtime ($/chip) 110 120 130 Regular operation time (hr) 2400 2400 2400 Overtime (hr) 720 720 720

There are no chips in stock at the beginning of the fi rst month. It takes 1.5 hr of production time to produce a chip and costs $5 to store a chip from one month to the next. Determine a production schedule that meets the demand requirements, does not exceed the monthly production time limitations, and minimizes cost. Note that no chips should be in stock at the end of the 3 months.

Mechanical/Aerospace Engineering 16.28 The total drag on an airfoil can be estimated by

D 5 0.01s V 2 1 0.95 s

aW V b2

friction lift

FIGURE P16.25 Torque transmitted to an inductor as a function of slip.

s

T

4 8 10

3

4

0 0

2

2

6

1

FIGURE P16.28 Plot of drag versus velocity for an airfoil.

400 800 1,200

10,000

20,000

Total

Minimum

Lift Friction

0 0

V

D

FIGURE P16.29 Roller bearings.

F

F

x

cha9792x_ch16_416-437.indd Page 436 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 436 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 437

and testing of mountain bikes (Fig. P16.33a). Suppose that you are given the task of predicting the horizontal and vertical displace- ment of a bike bracketing system in response to a force. Assume the forces you must analyze can be simplifi ed as depicted in Fig. P16.33b. You are interested in testing the response of the truss to a force exerted in any number of directions designated by the angle u. The parameters for the problem are E 5 Young’s modulus 5 231011 Pa, A 5 cross-sectional area 5 0.0001 m2, w 5 width 5 0.44 m, / 5 length 5 0.56 m, and h 5 height 5 0.5 m. The dis- placements x and y can be solved by determining the values that yield a minimum potential energy. Determine the displacements for a force of 10,000 N and a range of u’s from 08 (horizontal) to 908 (vertical).

The problem of fi nding the location of the maximum stress along the x axis can be shown to be equivalent to maximizing the function

f (x) 5 0.4

21 1 x2 2 21 1 x2 a1 2 0.4

1 1 x2 b 1 x

Find the x that maximizes f(x). 16.30 An aerospace company is developing a new fuel additive for commercial airliners. The additive is composed of three ingre- dients: X, Y, and Z. For peak performance, the total amount of additive must be at least 6 mL/L of fuel. For safety reasons, the sum of the highly fl ammable X and Y ingredients must not exceed 2.5 mL/L. In addition, the amount of the X ingredient must always be equal to or greater than the Y, and the Z must be greater than half the Y. If the cost per mL for the ingredients X, Y, and Z is 0.05, 0.025, and 0.15, respectively, determine the minimum cost mixture for each liter of fuel. 16.31 A manufacturing fi rm produces four types of automobile parts. Each is fi rst fabricated and then fi nished. The required worker hours and profi t for each part are

Part

A B C D

Fabrication time (hr/100 units) 2.5 1.5 2.75 2 Finishing time (hr/100 units) 3.5 3 3 2 Profi t ($/100 units) 375 275 475 325

The capacities of the fabrication and fi nishing shops over the next month are 640 and 960 hours, respectively. Determine how many of each part should be produced in order to maximize profi t. 16.32 In a similar fashion to the case study described in Sec. 16.4, develop the potential energy function for the system de- picted in Fig. P16.32. Develop contour and surface plots in MATLAB. Minimize the potential energy function in order to determine the equilibrium displacements x1 and x2 given the forcing function F 5 100 N, and the parameter ka 5 20 and kb 5 15 N/m. 16.33 Recent interest in competitive and recreational cycling has meant that engineers have directed their skills toward the design

FIGURE P16.32 Two frictionless masses connected to a wall by a pair of linear elastic springs.

ka kb F21

x2x1

FIGURE P16.33 (a) A mountain bike along with (b) a free-body diagram for a part of the frame.

(a)

x

Fy

h �

w

(b)

cha9792x_ch16_416-437.indd Page 437 28/10/13 8:06 PM f-479 cha9792x_ch16_416-437.indd Page 437 28/10/13 8:06 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

438

EPILOGUE: PART FOUR

The epilogues of other parts of this book contain a discussion and a tabular summary of the trade-offs among various methods as well as important formulas and relationships. Most of the methods of this part are quite complicated and, consequently, cannot be summarized with simple formulas and tabular summaries. Therefore, we deviate some- what here by providing the following narrative discussion of trade-offs and further refer- ences.

PT4.4 TRADE-OFFS

Chapter 13 dealt with fi nding the optimum of an unconstrained function of a single vari- able. The golden-section search method is a bracketing method requiring that an interval containing a single optimum be known. It has the advantage that it minimizes function evaluations and always converges. Parabolic interpolation also works best when imple- mented as a bracketing method, although it can also be programmed as an open method. However, in such cases, it may diverge. Both the golden-section search and parabolic interpolation do not require derivative evaluations. Thus, they are both appropriate meth- ods when the bracket can be readily defi ned and function evaluations are costly. Newton’s method is an open method not requiring that an optimum be bracketed. It can be implemented in a closed-form representation when fi rst and second derivatives can be determined analytically. It can also be implemented in a fashion similar to the secant method with fi nite-difference representations of the derivatives. Although Newton’s method converges rapidly near the optimum, it is often divergent for poor guesses. Convergence is also dependent on the nature of the function. Finally, hybrid approaches are available that orchestrate various methods to attain both reliability and effi ciency. Brent’s method does this by combining the reliable golden- section search with speedy parabolic interpolation. Chapter 14 covered two general types of methods to solve multidimensional uncon- strained optimization problems. Direct methods such as random searches and univariate searches do not require the evaluation of the function’s derivatives and are often ineffi - cient. However they also provide a tool to fi nd global rather than local optima. Pattern search methods like Powell’s method can be very effi cient and also do not require de- rivative evaluation. Gradient methods use either fi rst and sometimes second derivatives to fi nd the op- timum. The method of steepest ascent/descent provides a reliable but sometimes slow approach. In contrast, Newton’s method often converges rapidly when in the vicinity of a root, but sometimes suffers from divergence. The Marquardt method uses the steepest descent method at the starting location far away from the optimum and switches to Newton’s method near the optimum in an attempt to take advantage of the strengths of each method.

cha9792x_ep04_438-439.indd Page 438 29/10/13 9:52 PM F-468 cha9792x_ep04_438-439.indd Page 438 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT4.5 ADDITIONAL REFERENCES 439

The Newton method can be computationally costly because it requires computation of both the gradient vector and the Hessian matrix. Quasi-Newton approaches attempt to circumvent these problems by using approximations to reduce the number of matrix evaluations (particularly the evaluation, storage, and inversion of the Hessian). Research investigations continue today that explore the characteristics and respective strengths of various hybrid and tandem methods. Some examples are the Fletcher-Reeves conjugate gradient method and Davidon-Fletcher-Powell quasi-Newton methods. Chapter 15 was devoted to constrained optimization. For linear problems, linear pro- gramming based on the simplex method provides an effi cient means to obtain solutions. Approaches such as the GRG method are available to solve nonlinear constrained problems. Software packages include a wide variety of optimization capabilities. As described in Chap. 15, Excel, MATLAB software, and Mathcad all have built-in search capabilities that can be used for both one-dimensional and multidimensional problems routinely encountered in engineering and science.

PT4.5 ADDITIONAL REFERENCES

General overviews of optimization including some algorithms can be found in Press et al. (2007) and Moler (2004). For multidimensional problems, additional information can be found in Dennis and Schnabel (1996), Fletcher (1980, 1981), Gill et al. (1981), and Luenberger (1984). In addition, there are a number of advanced methods that are well suited for specifi c problem contexts. For example, genetic algorithms use strategies inspired by evolutionary biology such as inheritance, mutation, and selection. Because they do not make assump- tions regarding the underlying search space, such evolutionary algorithms are often use- ful for large problems with many local optima. Related techniques include simulated annealing and Tabu search. Hillier and Lieberman (2005) provide overviews of these and a number of other advanced techniques.

cha9792x_ep04_438-439.indd Page 439 29/10/13 9:52 PM F-468 cha9792x_ep04_438-439.indd Page 439 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART FIVE

cha9792x_p05_440-455.indd Page 440 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 440 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

441

PT5.1 MOTIVATION

Data are often given for discrete values along a continuum. However, you may require estimates at points between the discrete values. The present part of this book describes techniques to fi t curves to such data to obtain intermediate estimates. In addition, you may require a simplifi ed version of a complicated function. One way to do this is to compute values of the function at a number of discrete values along the range of interest. Then, a simpler function may be derived to fi t these values. Both of these applications are known as curve fi tting. There are two general approaches for curve fi tting that are distinguished from each other on the basis of the amount of error associated with these data. First, where these data exhibit a signifi cant degree of error or “noise,” the strategy is to derive a single curve that represents the general trend of these data. Because any individual data point may be incorrect, we make no effort to intersect every point. Rather, the curve is designed to follow the pattern of the points taken as a group. One approach of this nature is called least-squares regression (Fig. PT5.1a). Second, where these data are known to be very precise, the basic approach is to fi t a curve or a series of curves that pass directly through each of the points. Such data usually originate from tables. Examples are values for the density of water or for the heat capacity of gases as a function of temperature. The estimation of values between well-known discrete points is called interpolation (Fig. PT5.1b and c).

PT5.1.1 Noncomputer Methods for Curve Fitting

The simplest method for fi tting a curve to data is to plot the points and then sketch a line that visually conforms to these data. Although this is a valid option when quick estimates are required, the results are dependent on the subjective viewpoint of the per- son sketching the curve. For example, Fig. PT5.1 shows sketches developed from the same set of data by three engineers. The fi rst did not attempt to connect the points, but rather, characterized the general upward trend of these data with a straight line (Fig. PT5.1a). The second engineer used straight-line segments or linear interpolation to connect the points (Fig.  PT5.1b). This is a very common practice in engineering. If the values are truly close to being linear or are spaced closely, such an approximation provides estimates that are adequate for many engineering calculations. However, where the underlying relationship is highly curvilinear or these data are widely spaced, signifi cant errors can be introduced by such linear interpolation. The third engineer used curves to try to capture the meanderings suggested by these data (Fig. PT5.1c). A fourth or fi fth engineer would likely develop alternative fi ts. Obviously, our goal here is to develop systematic and objective methods for the purpose of deriving such curves.

CURVE FITTING

cha9792x_p05_440-455.indd Page 441 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 441 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

442 CURVE FITTING

PT5.1.2 Curve Fitting and Engineering Practice

Your fi rst exposure to curve fi tting may have been to determine intermediate values from tabulated data—for instance, from interest tables for engineering economics or from steam tables for thermodynamics. Throughout the remainder of your career, you will have frequent occasion to estimate intermediate values from such tables. Although many of the widely used engineering properties have been tabulated, there are a great many more that are not available in this convenient form. Special cases and new problem contexts often require that you measure your own data and develop your own predictive relationships. Two types of applications are generally encountered when fi tting experimental data: trend analysis and hypothesis testing. Trend analysis represents the process of using the pattern of these data to make predictions. For cases where these data are measured with high precision, you might

FIGURE PT5.1 Three attempts to fi t a “best” curve through fi ve data points. (a) Least-squares regression, (b) linear interpolation, and (c) curvilinear interpolation.

f (x)

x(a)

f (x)

x(b)

f (x)

x(c)

cha9792x_p05_440-455.indd Page 442 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 442 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.2 MATHEMATICAL BACKGROUND 443

utilize interpolating polynomials. Imprecise data are often analyzed with least-squares regression. Trend analysis may be used to predict or forecast values of the dependent variable. This can involve extrapolation beyond the limits of the observed data or interpolation within the range of the data. All fi elds of engineering commonly involve problems of this type. A second engineering application of experimental curve fi tting is hypothesis testing. Here, an existing mathematical model is compared with measured data. If the model coeffi cients are unknown, it may be necessary to determine values that best fi t the ob- served data. On the other hand, if estimates of the model coeffi cients are already avail- able, it may be appropriate to compare predicted values of the model with observed values to test the adequacy of the model. Often, alternative models are compared and the “best” one is selected on the basis of empirical observations. In addition to the above engineering applications, curve fi tting is important in other numerical methods such as integration and the approximate solution of differential equa- tions. Finally, curve-fi tting techniques can be used to derive simple functions to ap- proximate complicated functions.

PT5.2 MATHEMATICAL BACKGROUND

The prerequisite mathematical background for interpolation is found in the material on Taylor series expansions and fi nite divided differences introduced in Chap. 4. Least-squares regression requires additional information from the fi eld of statistics. If you are familiar with the concepts of the mean, standard deviation, residual sum of the squares, normal distribution, and confi dence intervals, feel free to skip the follow- ing pages and proceed directly to PT5.3. If you are unfamiliar with these concepts or are in need of a review, the following material is designed as a brief introduction to these topics.

PT5.2.1 Simple Statistics

Suppose that in the course of an engineering study, several measurements were made of a particular quantity. For example, Table PT5.1 contains 24 readings of the coeffi cient of thermal expansion of a structural steel. Taken at face value, these data provide a limited amount of information—that is, that the values range from a minimum of 6.395 to a maximum of 6.775. Additional insight can be gained by summarizing these data in one or more well-chosen statistics that convey as much information as possible about specifi c characteristics of the data set. These descriptive statistics are most often selected

TABLE PT5.1 Measurements of the coeffi cient of thermal expansion of structural steel [3 1026 in/(in ? 8F)].

6.495 6.595 6.615 6.635 6.485 6.555 6.665 6.505 6.435 6.625 6.715 6.655 6.755 6.625 6.715 6.575 6.655 6.605 6.565 6.515 6.555 6.395 6.775 6.685

cha9792x_p05_440-455.indd Page 443 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 443 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

444 CURVE FITTING

to represent (1) the location of the center of the distribution of these data and (2) the degree of spread of the data set. The most common location statistic is the arithmetic mean. The arithmetic mean (y) of a sample is defi ned as the sum of the individual data points (yi) divided by the num- ber of points (n), or

y 5 gyi n

(PT5.1)

where the summation (and all the succeeding summations in this introduction) is from i 5 1 through n. The most common measure of spread for a sample is the standard deviation (sy) about the mean,

sy 5 B

St n 2 1

(PT5.2)

where St is the total sum of the squares of the residuals between the data points and the mean, or

St 5 g (yi 2 y) 2 (PT5.3)

Thus, if the individual measurements are spread out widely around the mean, St (and, consequently, sy) will be large. If they are grouped tightly, the standard deviation will be small. The spread can also be represented by the square of the standard deviation, which is called the variance:

s2y 5 g (yi 2 y)

2

n 2 1 (PT5.4)

Note that the denominator in both Eqs. (PT5.2) and (PT5.4) is n 2 1. The quantity n 2 1 is referred to as the degrees of freedom. Hence St and sy are said to be based on n 2 1 degrees of freedom. This nomenclature derives from the fact that the sum of the quanti- ties upon which St is based (that is, y 2 y1, y 2 y2, p , y 2 yn) is zero. Consequently, if y is known and n 2 1 of the values are specifi ed, the remaining value is fi xed. Thus, only n 2 1 of the values are said to be freely determined. Another justifi cation for divid- ing by n 2 1 is the fact that there is no such thing as the spread of a single data point. For the case where n 5 1, Eqs. (PT5.2) and (PT5.4) yield a meaningless result of infi nity. It should be noted that an alternative, more convenient formula is available to com- pute the standard deviation,

s2y 5 gy2i 2 ( gyi)

2yn n 2 1

This version does not require precomputation of y and yields an identical result as Eq. (PT5.4).

cha9792x_p05_440-455.indd Page 444 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 444 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.2 MATHEMATICAL BACKGROUND 445

A fi nal statistic that has utility in quantifying the spread of data is the coeffi cient of variation (c.v.). This statistic is the ratio of the standard deviation to the mean. As such, it provides a normalized measure of the spread. It is often multiplied by 100 so that it can be expressed in the form of a percent:

c.v. 5 sy

y 100% (PT5.5)

Notice that the coeffi cient of variation is similar in spirit to the percent relative error (�t) discussed in Sec. 3.3. That is, it is the ratio of a measure of error (sy) to an estimate of the true value (y).

EXAMPLE PT5.1 Simple Statistics of a Sample

Problem Statement. Compute the mean, variance, standard deviation, and coeffi cient of variation for the data in Table PT5.1.

TABLE PT5.2 Computations for statistics for the readings of the coeffi cient of thermal expansion. The frequencies and bounds are developed to construct the histogram in Fig. PT5.2.

Interval

Lower Upper i yi (yi 2 yw)

2 Frequency Bound Bound

1 6.395 0.042025 1 6.36 6.40 2 6.435 0.027225 1 6.40 6.44 3 6.485 0.013225 4 6.495 0.011025 4 6.48 6.52 5 6.505 0.009025 6 6.515 0.007225 7 6.555 0.002025 8 6.555 0.002025 2 6.52 6.56

9 6.565 0.001225 10 6.575 0.000625 3 6.56 6.60 11 6.595 0.000025 12 6.605 0.000025 13 6.615 0.000225 14 6.625 0.000625 5 6.60 6.64 15 6.625 0.000625 16 6.635 0.001225 17 6.655 0.003025 18 6.655 0.003025 3 6.64 6.68 19 6.665 0.004225 20 6.685 0.007225 21 6.715 0.013225 3 6.68 6.72 22 6.715 0.013225 23 6.755 0.024025 1 6.72 6.76 24 6.775 0.030625 1 6.76 6.80

S 158.4 0.217000

cha9792x_p05_440-455.indd Page 445 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 445 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

446 CURVE FITTING

Solution. These data are added (Table PT5.2), and the results are used to compute [Eq. (PT5.1)]

y 5 158.4

24 5 6.6

As in Table PT5.2, the sum of the squares of the residuals is 0.217000, which can be used to compute the standard deviation [Eq. (PT5.2)]:

sy 5 B

0.217000

24 2 1 5 0.097133

the variance [Eq. (PT5.4)]:

s2y 5 0.009435

and the coeffi cient of variation [Eq. (PT5.5)]:

c.v. 5 0.097133

6.6 100% 5 1.47%

PT5.2.2 The Normal Distribution

Another characteristic that bears on the present discussion is the data distribution—that is, the shape with which these data are spread around the mean. A histogram provides a simple visual representation of the distribution. As seen in Table PT5.2, the histogram is constructed by sorting the measurements into intervals. The units of measurement are plot- ted on the abscissa and the frequency of occurrence of each interval is plotted on the or- dinate. Thus, fi ve of the measurements fall between 6.60 and 6.64. As in Fig. PT5.2, the histogram suggests that most of these data are grouped close to the mean value of 6.6. If we have a very large set of data, the histogram often can be approximated by a smooth curve. The symmetric, bell-shaped curve superimposed on Fig. PT5.2 is one such characteristic shape—the normal distribution. Given enough additional measurements, the histogram for this particular case could eventually approach the normal distribution. The concepts of the mean, standard deviation, residual sum of the squares, and normal distribution all have great relevance to engineering practice. A very simple ex- ample is their use to quantify the confi dence that can be ascribed to a particular measure- ment. If a quantity is normally distributed, the range defi ned by y 2 sy to y 1 sy will encompass approximately 68 percent of the total measurements. Similarly, the range defi ned by y 2 2sy to y 1 2sy will encompass approximately 95 percent. For example, for the data in Table PT5.1 (y 5 6.6 and sy 5 0.097133), we can make the statement that approximately 95 percent of the readings should fall between 6.405734 and 6.794266. If someone told us that they had measured a value of 7.35, we would suspect that the measurement might be erroneous. The following section elaborates on such evaluations.

PT5.2.3 Estimation of Confi dence Intervals

As should be clear from the previous sections, one of the primary aims of statistics is to estimate the properties of a population based on a limited sample drawn from that

cha9792x_p05_440-455.indd Page 446 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 446 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.2 MATHEMATICAL BACKGROUND 447

population. Clearly, it is impossible to measure the coeffi cient of thermal expansion for every piece of structural steel that has ever been produced. Consequently, as seen in Tables PT5.1 and PT5.2, we can randomly make a number of measurements and, on the basis of the sample, attempt to characterize the properties of the entire population. Because we “infer” properties of the unknown population from a limited sample, the endeavor is called statistical inference. Because the results are often reported as estimates of the population parameters, the process is also referred to as estimation. We have already shown how we estimate the central tendency (sample mean, y) and spread (sample standard deviation and variance) of a limited sample. Now, we will briefl y describe how we can attach probabilistic statements to the quality of these estimates. In particular, we will discuss how we can defi ne a confi dence interval around our estimate of the mean. We have chosen this particular topic because of its direct relevance to the regression models we will be describing in Chap. 17. Note that in the following discussion, the nomenclature y and sy refer to the sample mean and standard deviation, respectively. The nomenclature � and � refer to the popu- lation mean and standard deviation, respectively. The former are sometimes referred to as the “estimated” mean and standard deviation, whereas the latter are sometimes called the “true” mean and standard deviation. An interval estimator gives the range of values within which the parameter is ex- pected to lie with a given probability. Such intervals are described as being one-sided or two-sided. As the name implies, a one-sided interval expresses our confi dence that the parameter estimate is less than or greater than the true value. In contrast, the two-sided interval deals with the more general proposition that the estimate agrees with the truth with no consideration to the sign of the discrepancy. Because it is more general, we will focus on the two-sided interval.

FIGURE PT5.2 A histogram used to depict the distribution of data. As the number of data points increases, the histogram could approach the smooth, bell-shaped curve called the normal distribution.

5

4

Fr eq

u en

cy

3

2

1

6.4 6.6 6.8 0

cha9792x_p05_440-455.indd Page 447 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 447 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

448 CURVE FITTING

A two-sided interval can be described by the statement

P{L # m # U} 5 1 2 a

which reads, “the probability that the true mean of y, �, falls within the bound from L  to U is 1 2 �.” The quantity � is called the signifi cance level. So the problem of defi ning a confi dence interval reduces to estimating L and U. Although it is not abso- lutely necessary, it is customary to view the two-sided interval with the � probability distributed evenly as �y2 in each tail of the distribution, as in Fig. PT5.3. If the true variance of the distribution of y, �2, is known (which is not usually the case), statistical theory states that the sample mean y comes from a normal distribution with mean � and variance �2yn (Box PT5.1). In the case illustrated in Fig. PT5.3, we really do not know �. Therefore, we do not know where the normal curve is exactly located with respect to y. To circumvent this dilemma, we compute a new quantity, the standard normal estimate

z 5 y 2 m

sy2n (PT5.6)

which represents the normalized distance between y and �. According to statistical theory, this quantity should be normally distributed with a mean of 0 and a variance of 1. Furthermore, the probability that z would fall within the unshaded region of Fig. PT5.3

FIGURE PT5.3 A two-sided confi dence interval. The abscissa scale in (a) is written in the natural units of the ran- dom variable y. The normalized version of the abscissa in (b) has the mean at the origin and scales the axis so that the standard deviation corresponds to a unit value.

L

�/2

1 – �

Distribution of means of y, y–

� U

y (a)

z– �/2 – 1 10 z�/2

z– (b)

�/2

�– �

cha9792x_p05_440-455.indd Page 448 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 448 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.2 MATHEMATICAL BACKGROUND 449

should be 1 2 �. Therefore, the statement can be made that

y 2 m

sy1n , 2zay2  or  

y 2 m

sy1n . zay2

with a probability of �. The quantity z�y2 is a standard normal random variable. This is the distance measured along the normalized axis above and below the mean that encompasses 1 2 � probability (Fig. PT5.3b). Values of z�y2 are tabulated in statistics books (for example, Milton and Arnold, 2002). They can also be calculated using functions on software packages like Excel, MATLAB, and Mathcad. As an example, for � 5 0.05 (in other words, defi ning an interval encompassing 95%), z�y2 is equal to about 1.96. This means that an interval around the mean of width 61.96 times the standard deviation will encompass approxi- mately 95% of the distribution. These results can be rearranged to yield

L # m # U

Box PT5.1 A Little Statistics

Most engineers take several courses to become profi cient at statis- tics. Because you may not have taken such a course yet, we would like to mention a few ideas that might make this present section more coherent. As we have stated, the “game” of inferential statistics assumes that the random variable you are sampling, y, has a true mean (�) and variance (�2). Further, in the present discussion, we also as- sume that it has a particular distribution: the normal distribution. The variance of this normal distribution has a fi nite value that spec- ifi es the “spread” of the normal distribution. If the variance is large, the distribution is broad. Conversely, if the variance is small, the distribution is narrow. Thus, the true variance quantifi es the intrin- sic uncertainty of the random variable. In the game of statistics, we take a limited number of measure- ments of this quantity called a sample. From this sample, we can compute an estimated mean (y) and variance (s2y). The more mea- surements we take, the better the estimates approximate the true values. That is, as nS `, yS m and s2yS s2. Suppose that we take n samples and compute an estimated mean y1. Then, we take another n samples and compute another, y2. We can keep repeating this process until we have generated a sample of means: y1, y2, y3, p , ym, where m is large. We can then develop a histogram of these means and determine a “distribution of the means,” as well as a “mean of the means” and a “standard deviation of the means.” Now the question arises: does this new distribution of means and its statistics behave in a predictable fashion?

There is an extremely important theorem known as the Central Limit Theorem that speaks directly to this question. It can be stated as

Let y1, y2, . . . , yn be a random sample of size n from a distribu- tion with mean � and variance �2. Then, for large n, y is approxi- mately normal with mean � and variance �2yn. Furthermore, for large n, the random variable (y 2 m)y(sy1n) is approximately standard normal.

Thus, the theorem states the remarkable result that the distri- bution of means will always be normally distributed regardless of the underlying distribution of the random variables! It also yields the expected result that given a suffi ciently large sample, the mean of the means should converge on the true population mean �. Further, the theorem says that as the sample size gets larger, the variance of the means should approach zero. This makes sense, because if n is small, our individual estimates of the mean should be poor and the variance of the means should be large. As n in- creases, our estimates of the mean will improve and hence their spread should shrink. The Central Limit Theorem neatly defi nes exactly how this shrinkage relates to both the true variance and the sample size, that is, as �2yn. Finally, the theorem states the important result that we have given as Eq. (PT5.6). As is shown in this section, this result is the basis for constructing confi dence intervals for the mean.

cha9792x_p05_440-455.indd Page 449 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 449 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

450 CURVE FITTING

with a probability of 1 2 �, where

L 5 y 2 s

1n zay2  U 5 y 1

s

1n zay2 (PT5.7)

Now, although the foregoing provides an estimate of L and U, it is based on knowl- edge of the true variance �. For our case, we know only the estimated variance sy. A straightforward alternative would be to develop a version of Eq. (PT5.6) based on sy,

t 5 y 2 m

syy1n (PT5.8)

Even when we sample from a normal distribution, this fraction will not be normally distributed, particularly when n is small. It was found by W. S. Gossett that the random variable defi ned by Eq. (PT5.8) follows the so-called Student-t, or simply, t distribution. For this case,

L 5 y 2 sy

1n tay2, n21  U 5 y 1

sy

1n tay2, n21 (PT5.9)

where t�y2, n21 is the standard random variable for the t distribution for a probability of �y2. As was the case for z�y2, values are tabulated in statistics books and can also be calculated using software packages and libraries. For example, if � 5 0.05 and n 5 20, t�y2, n21 5 2.086. The t distribution can be thought of as a modifi cation of the normal distribution that accounts for the fact that we have an imperfect estimate of the standard deviation. When n is small, it tends to be fl atter than the normal (see Fig. PT5.4). Therefore, for small

FIGURE PT5.4 Comparison of the normal distribution with the t distribution for n 5 3 and n 5 6. Notice how the t distribution is generally fl atter.

– 1– 2– 3 0 Z or t

21 3

t(n = 6)

t(n = 3)

Normal

cha9792x_p05_440-455.indd Page 450 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 450 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.2 MATHEMATICAL BACKGROUND 451

numbers of measurements, it yields wider and hence more conservative confi dence in- tervals. As n grows larger, the t distribution converges on the normal.

EXAMPLE PT5.2 Confi dence Interval on the Mean

Problem Statement. Determine the mean and the corresponding 95% confi dence interval for the data from Table PT5.1. Perform three estimates based on (a) the fi rst 8, (b) the fi rst 16, and (c) all 24 measurements.

Solution. (a) The mean and standard deviation for the fi rst 8 points is

y 5 52.72

8 5 6.59  sy 5 B

347.4814 2 (52.72)2y8 8 2 1

5 0.089921

The appropriate t statistic can be calculated as

t0.05y2, 821 5 t0.025, 7 5 2.364623

which can be used to compute the interval

L 5 6.59 2 0.089921

18 2.364623 5 6.5148

U 5 6.59 1 0.089921

18 2.364623 5 6.6652

or

6.5148 # m # 6.6652

FIGURE PT5.5 Estimates of the mean and 95% confi dence intervals for different numbers of sample size.

6.606.556.50 Coefficient of thermal expansion [� 10– 6 in/(in • �F)]

6.706.65

n = 24

n = 16

y–

n = 8

cha9792x_p05_440-455.indd Page 451 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 451 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

452 CURVE FITTING

Thus, based on the fi rst eight measurements, we conclude that there is a 95% probabil- ity that the true mean falls within the range 6.5148 to 6.6652. The two other cases for (b) 16 points and (c) 24 points can be calculated in a similar fashion and the results tabulated along with case (a) as

n yw sy t�y2,n21 L U

8 6.5900 0.089921 2.364623 6.5148 6.6652 16 6.5794 0.095845 2.131451 6.5283 6.6304 24 6.6000 0.097133 2.068655 6.5590 6.6410

These results, which are also summarized in Fig. PT5.5, indicate the expected outcome that the confi dence interval becomes more narrow as n increases. Thus, the more mea- surements we take, our estimate of the true value becomes more refi ned.

The above is just one simple example of how statistics can be used to make judg- ments regarding uncertain data. These concepts will also have direct relevance to our discussion of regression models. You can consult any basic statistics book (for example, Milton and Arnold, 2002) to obtain additional information on the subject.

PT5.3 ORIENTATION

Before we proceed to numerical methods for curve fi tting, some orientation might be helpful. The following is intended as an overview of the material discussed in Part Five. In addition, we have formulated some objectives to help focus your efforts when study- ing the material.

PT5.3.1 Scope and Preview

Figure PT5.6 provides a visual overview of the material to be covered in Part Five. Chapter 17 is devoted to least-squares regression. We will fi rst learn how to fi t the “best” straight line through a set of uncertain data points. This technique is called lin- ear regression. Besides discussing how to calculate the slope and intercept of this straight line, we also present quantitative and visual methods for evaluating the validity of the results. In addition to fi tting a straight line, we also present a general technique for fi tting a “best’’ polynomial. Thus, you will learn to derive a parabolic, cubic, or higher-order polynomial that optimally fi ts uncertain data. Linear regression is a subset of this more general approach, which is called polynomial regression. The next topic covered in Chap. 17 is multiple linear regression. It is designed for the case where the dependent variable y is a linear function of two or more independent variables x1, x2, . . . , xm. This approach has special utility for evaluating experimental data where the variable of interest is dependent on a number of different factors.

cha9792x_p05_440-455.indd Page 452 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 452 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.3 ORIENTATION 453

After multiple regression, we illustrate how polynomial and multiple regression are both subsets of a general linear least-squares model. Among other things, this will allow us to introduce a concise matrix representation of regression and discuss its general statistical properties.

FIGURE PT5.6 Schematic of the organization of the material in Part Five: Curve Fitting.

PART 5 Curve Fitting

CHAPTER 20 Case Studies

EPILOGUE

18.6 Splines

18.7 Multidimensional

interpolation

18.5 Additional comments

18.4 Inverse

interpolation

18.3 Polynomial coefficients

18.2 Lagrange

polynomial

18.1 Newton

polynomial

PT 5.2 Mathematical background

PT 5.6 Advanced methods

PT 5.5 Important formulas

20.4 Mechanical engineering

20.3 Electrical

engineering

20.2 Civil

engineering

20.1 Chemical

engineering

19.8 Software packages

19.7 Power

spectrum

19.1 Sinusoids

19.2 Continuous

Fourier series

19.6 Fast Fourier transform

19.5 Discrete Fourier

transform

19.3 Frequency and time domains

19.4 Fourier

transform

PT 5.4 Trade-offs

PT 5.3 Orientation

PT 5.1 Motivation

17.2 Polynomial regression

17.3 Multiple

regression

17.4 General linear least squares

17.5 Nonlinear regression

17.1 Linear

regression

CHAPTER 17 Least-Squares

Regression

CHAPTER 19 Fourier

Approximation

CHAPTER 18 Interpolation

cha9792x_p05_440-455.indd Page 453 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 453 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

454 CURVE FITTING

Finally, the last sections of Chap. 17 are devoted to nonlinear regression. This ap- proach is designed to compute a least-squares fi t of a nonlinear equation to data. In Chap. 18, the alternative curve-fi tting technique called interpolation is de- scribed. As discussed previously, interpolation is used for estimating intermediate values between precise data points. In Chap. 18, polynomials are derived for this purpose. We introduce the basic concept of polynomial interpolation by using straight lines and parabolas to connect points. Then, we develop a generalized procedure for fi tting an nth-order polynomial. Two formats are presented for expressing these poly- nomials in equation form. The fi rst, called Newton’s interpolating polynomial, is pref- erable when the appropriate order of the polynomial is unknown. The second, called the Lagrange interpolating polynomial, has advantages when the proper order is known beforehand. The next section of Chap. 18 presents an alternative technique for fi tting precise data points. This technique, called spline interpolation, fi ts polynomials to data but in a piece- wise fashion. As such, it is particularly well-suited for fi tting data that are generally smooth but exhibit abrupt local changes. Finally, we provide a brief introduction to multidimensional interpolation. Chapter 19 deals with the Fourier transform approach to curve fi tting where periodic functions are fi t to data. Our emphasis in this section will be on the fast Fourier trans- form. At the end of this chapter, we also include an overview of several software pack- ages that can be used for curve fi tting. These are Excel, MATLAB, and Mathcad. Chapter 20 is devoted to engineering applications that illustrate the utility of the numerical methods in engineering problem contexts. Examples are drawn from the four major specialty areas of chemical, civil, electrical, and mechanical engineering. In addi- tion, some of the applications illustrate how software packages can be applied for engi- neering problem solving. Finally, an epilogue is included at the end of Part Five. It contains a summary of the important formulas and concepts related to curve fi tting as well as a discussion of trade-offs among the techniques and suggestions for future study.

PT5.3.2 Goals and Objectives

Study Objectives. After completing Part Five, you should have greatly enhanced your capability to fi t curves to data. In general, you should have mastered the techniques, have learned to assess the reliability of the answers, and be capable of choosing the preferred method (or methods) for any particular problem. In addition to these general goals, the specifi c concepts in Table PT5.3 should be assimilated and mastered.

Computer Objectives. You have been provided with simple computer algorithms to implement the techniques discussed in Part Five. You may also have access to software packages and libraries. All have utility as learning tools. Pseudocode algorithms are provided for most of the methods in Part Five. This information will allow you to expand your software library to include techniques beyond polynomial regression. For example, you may fi nd it useful from a professional view- point to have software to implement multiple linear regression, Newton’s interpolating polynomial, cubic spline interpolation, and the fast Fourier transform.

cha9792x_p05_440-455.indd Page 454 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 454 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.3 ORIENTATION 455

In addition, one of your most important goals should be to master several of the general-purpose software packages that are widely available. In particular, you should become adept at using these tools to implement numerical methods for engineering problem solving.

TABLE PT5.3 Specifi c study objectives for Part Five.

1. Understand the fundamental difference between regression and interpolation and realize why confusing the two could lead to serious problems

2. Understand the derivation of linear least-squares regression and be able to assess the reliability of the fi t using graphical and quantitative assessments

3. Know how to linearize data by transformation 4. Understand situations where polynomial, multiple, and nonlinear regression are appropriate 5. Be able to recognize general linear models, understand the general matrix formulation of linear least

squares, and know how to compute confi dence intervals for parameters 6. Understand that there is one and only one polynomial of degree n or less that passes exactly

through n 1 1 points 7. Know how to derive the fi rst-order Newton’s interpolating polynomial 8. Understand the analogy between Newton’s polynomial and the Taylor series expansion and how it

relates to the truncation error 9. Recognize that the Newton and Lagrange equations are merely different formulations of the same

interpolating polynomial and understand their respective advantages and disadvantages 10. Realize that more accurate results are generally obtained if data used for interpolation are centered

around and close to the unknown point 11. Realize that data points do not have to be equally spaced nor in any particular order for either the

Newton or Lagrange polynomials 12. Know why equispaced interpolation formulas have utility 13. Recognize the liabilities and risks associated with extrapolation 14. Understand why spline functions have utility for data with local areas of abrupt change 15. Understand how interpolating polynomials can be applied in two dimensions 16. Recognize how the Fourier series is used to fi t data with periodic functions 17. Understand the difference between the frequency and time domains

cha9792x_p05_440-455.indd Page 455 29/10/13 9:54 PM F-468 cha9792x_p05_440-455.indd Page 455 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17 C H A P T E R 17

456

Least-Squares Regression

Where substantial error is associated with data, polynomial interpolation is inappropriate and may yield unsatisfactory results when used to predict intermediate values. Experi- mental data are often of this type. For example, Fig. 17.1a shows seven experimentally derived data points exhibiting signifi cant variability. Visual inspection of these data sug- gests a positive relationship between y and x. That is, the overall trend indicates that higher values of y are associated with higher values of x. Now, if a sixth-order interpo- lating polynomial is fi tted to these data (Fig. 17.1b), it will pass exactly through all of the points. However, because of the variability in these data, the curve oscillates widely in the interval between the points. In particular, the interpolated values at x 5 1.5 and x 5 6.5 appear to be well beyond the range suggested by these data. A more appropriate strategy for such cases is to derive an approximating function that fi ts the shape or general trend of the data without necessarily matching the indi- vidual points. Figure 17.1c illustrates how a straight line can be used to generally char- acterize the trend of these data without passing through any particular point. One way to determine the line in Fig. 17.1c is to visually inspect the plotted data and then sketch a “best” line through the points. Although such “eyeball” approaches have commonsense appeal and are valid for “back-of-the-envelope” calculations, they are defi cient because they are arbitrary. That is, unless the points defi ne a perfect straight line (in which case, interpolation would be appropriate), different analysts would draw different lines. To remove this subjectivity, some criterion must be devised to establish a basis for the fi t. One way to do this is to derive a curve that minimizes the discrepancy between the data points and the curve. A technique for accomplishing this objective, called least- squares regression, will be discussed in the present chapter.

17.1 LINEAR REGRESSION

The simplest example of a least-squares approximation is fi tting a straight line to a set of paired observations: (x1, y1), (x2, y2), . . . , (xn, yn). The mathematical expression for the straight line is

y 5 a0 1 a1x 1 e (17.1)

cha9792x_ch17_456-489.indd Page 456 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 456 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 457

where a0 and a1 are coeffi cients representing the intercept and the slope, respectively, and e is the error, or residual, between the model and the observations, which can be represented by rearranging Eq. (17.1) as

e 5 y 2 a0 2 a1x

Thus, the error, or residual, is the discrepancy between the true value of y and the ap- proximate value, a0 1 a1x, predicted by the linear equation.

y

x

(a)

5

50 0

y

x

(b)

5

50 0

y

x

(c)

5

50 0

FIGURE 17.1 (a) Data exhibiting signifi cant error. (b) Polynomial fi t oscillating beyond the range of the data. (c) More satisfactory result using the least-squares fi t.

cha9792x_ch17_456-489.indd Page 457 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 457 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

458 LEAST-SQUARES REGRESSION

17.1.1 Criteria for a “Best” Fit

One strategy for fi tting a “best” line through the data would be to minimize the sum of the residual errors for all the available data, as in

a n

i51 ei 5 a

n

i51 (yi 2 a0 2 a1 xi) (17.2)

where n 5 total number of points. However, this is an inadequate criterion, as illustrated by Fig. 17.2a which depicts the fi t of a straight line to two points. Obviously, the best

FIGURE 17.2 Examples of some criteria for “best fi t” that are inadequate for regression: (a) minimizes the sum of the residuals, (b) minimizes the sum of the absolute values of the residuals, and (c) minimizes the maximum error of any individual point.

y

Midpoint

Outlier

x

(a)

y

x

(b)

y

x

(c)

cha9792x_ch17_456-489.indd Page 458 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 458 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 459

fi t is the line connecting the points. However, any straight line passing through the mid- point of the connecting line (except a perfectly vertical line) results in a minimum value of Eq. (17.2) equal to zero because the errors cancel. Therefore, another logical criterion might be to minimize the sum of the absolute values of the discrepancies, as in

a n

i51 Zei Z 5 a

n

i51 Zyi 2 a0 2 a1xi Z

Figure 17.2b demonstrates why this criterion is also inadequate. For the four points shown, any straight line falling within the dashed lines will minimize the sum of the absolute values. Thus, this criterion also does not yield a unique best fi t. A third strategy for fitting a best line is the minimax criterion. In this technique, the line is chosen that minimizes the maximum distance that an individual point falls from the line. As depicted in Fig. 17.2c, this strategy is ill-suited for regres- sion because it gives undue influence to an outlier, that is, a single point with a large error. It should be noted that the minimax principle is sometimes well-suited for fitting a simple function to a complicated function (Carnahan, Luther, and Wilkes, 1969). A strategy that overcomes the shortcomings of the aforementioned approaches is to minimize the sum of the squares of the residuals between the measured y and the y calculated with the linear model

Sr 5 a n

i51 e2i 5a

n

i51 (yi, measured 2 yi, model)

2 5a n

i51 (yi 2 a0 2 a1xi)

2 (17.3)

This criterion has a number of advantages, including the fact that it yields a unique line for a given set of data. Before discussing these properties, we will present a technique for determining the values of a0 and a1 that minimize Eq. (17.3).

17.1.2 Least-Squares Fit of a Straight Line

To determine values for a0 and a1, Eq. (17.3) is differentiated with respect to each coef- fi cient:

0Sr 0a0

5 22a (yi 2 a0 2 a1xi)

0Sr 0a1

5 22a [(yi 2 a0 2 a1xi)xi]

Note that we have simplifi ed the summation symbols; unless otherwise indicated, all summations are from i 5 1 to n. Setting these derivatives equal to zero will result in a minimum Sr. If this is done, the equations can be expressed as

0 5 a yi 2 a a0 2 a a1xi

0 5 a yi xi 2 a a0 xi 2 a a1x 2 i

cha9792x_ch17_456-489.indd Page 459 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 459 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

460 LEAST-SQUARES REGRESSION

Now, realizing that Sa0 5 na0, we can express the equations as a set of two simultane- ous linear equations with two unknowns (a0 and a1):

na0 1 (a xi)a1 5 a yi (17.4) (a xi)a0 1 (a x2i )a1 5 a xi yi (17.5)

These are called the normal equations. They can be solved simultaneously

a1 5 no xi yi 2 oxi oyi no x2i 2 (oxi)2

(17.6)

This result can then be used in conjunction with Eq. (17.4) to solve for

a0 5 y 2 a1x (17.7)

where y and x are the means of y and x, respectively.

EXAMPLE 17.1 Linear Regression

Problem Statement. Fit a straight line to the x and y values in the fi rst two columns of Table 17.1.

Solution. The following quantities can be computed:

n 5 7  a xi yi 5 119.5  a x 2 i 5 140

a xi 5 28  x 5 28

7 5 4

a yi 5 24  y 5 24

7 5 3.428571

Using Eqs. (17.6) and (17.7),

a1 5 7(119.5) 2 28(24)

7(140) 2 (28)2 5 0.8392857

a0 5 3.428571 2 0.8392857(4) 5 0.07142857

TABLE 17.1 Computations for an error analysis of the linear fi t.

xi yi (yi 2 y ) (yi 2 a0 2 a1xi )2

1 0.5 8.5765 0.1687 2 2.5 0.8622 0.5625 3 2.0 2.0408 0.3473 4 4.0 0.3265 0.3265 5 3.5 0.0051 0.5896 6 6.0 6.6122 0.7972 7 5.5 4.2908 0.1993 S 24.0 22.7143 2.9911

cha9792x_ch17_456-489.indd Page 460 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 460 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 461

17.1.3 Quantifi cation of Error of Linear Regression

Any line other than the one computed in Example 17.1 results in a larger sum of the squares of the residuals. Thus, the line is unique and in terms of our chosen criterion is a “best” line through the points. A number of additional properties of this fi t can be elucidated by examining more closely the way in which residuals were computed. Recall that the sum of the squares is defi ned as [Eq. (17.3)]

Sr 5 a n

i51 e

2 i 5 a

n

i51 (yi 2 a0 2 a1xi)

2 (17.8)

Notice the similarity between Eqs. (PT5.3) and (17.8). In the former case, the square of the residual represented the square of the discrepancy between the data and a single estimate of the measure of central tendency—the mean. In Eq. (17.8), the square of the residual represents the square of the vertical distance between the data and another mea- sure of central tendency—the straight line (Fig. 17.3). The analogy can be extended further for cases where (1) the spread of the points around the line is of similar magnitude along the entire range of the data and (2) the distribution of these points about the line is normal. It can be demonstrated that if these criteria are met, least-squares regression will provide the best (that is, the most likely) estimates of a0 and a1 (Draper and Smith, 1981). This is called the maximum likelihood

Therefore, the least-squares fi t is

y 5 0.07142857 1 0.8392857x

The line, along with the data, is shown in Fig. 17.1c.

FIGURE 17.3 The residual in linear regression represents the vertical distance between a data point and the straight line.

y

yi

xi

a0 + a1xi

Measurement

yi – a0 – a1xi

Re gre

ssi on

lin e

x

cha9792x_ch17_456-489.indd Page 461 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 461 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

462 LEAST-SQUARES REGRESSION

principle in statistics. In addition, if these criteria are met, a “standard deviation” for the regression line can be determined as [compare with Eq. (PT5.2)]

syyx 5 A

Sr n 2 2

(17.9)

where syyx is called the standard error of the estimate. The subscript notation “yyx” desig- nates that the error is for a predicted value of y corresponding to a particular value of x. Also, notice that we now divide by n 2 2 because two data-derived estimates—a0 and a1—were used to compute Sr; thus, we have lost two degrees of freedom. As with our discussion of the standard deviation in PT5.2.1, another justifi cation for dividing by n 2 2 is that there is no such thing as the “spread of data” around a straight line connecting two points. Thus, for the case where n 5 2, Eq. (17.9) yields a meaningless result of infi nity. Just as was the case with the standard deviation, the standard error of the estimate quantifi es the spread of the data. However, sy/x quantifi es the spread around the regression line as shown in Fig. 17.4b in contrast to the original standard deviation sy that quantifi ed the spread around the mean (Fig. 17.4a). The above concepts can be used to quantify the “goodness” of our fi t. This is par- ticularly useful for comparison of several regressions (Fig. 17.5). To do this, we return to the original data and determine the total sum of the squares around the mean for the dependent variable (in our case, y). As was the case for Eq. (PT5.3), this quantity is designated St. This is the magnitude of the residual error associated with the dependent variable prior to regression. After performing the regression, we can compute Sr, the sum of the squares of the residuals around the regression line. This characterizes the residual error that remains after the regression. It is, therefore, sometimes called the unexplained

FIGURE 17.4 Regression data showing (a) the spread of the data around the mean of the dependent variable and (b) the spread of the data around the best-fi t line. The reduction in the spread in going from (a) to (b), as indicated by the bell-shaped curves at the right, represents the improvement due to linear regression.

(a) (b)

cha9792x_ch17_456-489.indd Page 462 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 462 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 463

sum of the squares. The difference between the two quantities, St 2 Sr, quantifi es the improvement or error reduction due to describing the data in terms of a straight line rather than as an average value. Because the magnitude of this quantity is scale-dependent, the difference is normalized to St to yield

r2 5 St 2 Sr

St (17.10)

where r2 is called the coeffi cient of determination and r is the correlation coeffi cient (52r2). For a perfect fi t, Sr 5 0 and r 5 r2 5 1, signifying that the line explains 100 percent of the variability of the data. For r 5 r2 5 0, Sr 5 St and the fi t represents no improvement. An alternative formulation for r that is more convenient for computer implementation is

r 5 noxi yi 2 (oxi)(oyi)

2nox2i 2 (oxi) 22noy2i 2 (oyi)

2 (17.11)

y

x

(a)

y

x

(b)

FIGURE 17.5 Examples of linear regression with (a) small and (b) large residual errors.

cha9792x_ch17_456-489.indd Page 463 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 463 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

464 LEAST-SQUARES REGRESSION

EXAMPLE 17.2 Estimation of Errors for the Linear Least-Squares Fit

Problem Statement. Compute the total standard deviation, the standard error of the estimate, and the correlation coeffi cient for the data in Example 17.1.

Solution. The summations are performed and presented in Table 17.1. The standard deviation is [Eq. (PT5.2)]

sy 5 A

22.7143

7 2 1 5 1.9457

and the standard error of the estimate is [Eq. (17.9)]

syyx 5 A

2.9911

7 2 2 5 0.7735

Thus, because syyx , sy, the linear regression model has merit. The extent of the improve- ment is quantifi ed by [Eq. (17.10)]

r2 5 22.7143 2 2.9911

22.7143 5 0.868

or

r 5 10.868 5 0.932

These results indicate that 86.8 percent of the original uncertainty has been explained by the linear model.

Before proceeding to the computer program for linear regression, a word of caution is in order. Although the correlation coeffi cient provides a handy measure of goodness- of-fi t, you should be careful not to ascribe more meaning to it than is warranted. Just because r is “close” to 1 does not mean that the fi t is necessarily “good.” For example, it is possible to obtain a relatively high value of r when the underlying relationship between y and x is not even linear. Draper and Smith (1981) provide guidance and ad- ditional material regarding assessment of results for linear regression. In addition, at the minimum, you should always inspect a plot of the data along with your regression curve. As described in the next section, software packages include such a capability.

17.1.4 Computer Program for Linear Regression

It is a relatively trivial matter to develop a pseudocode for linear regression (Fig. 17.6). As mentioned above, a plotting option is critical to the effective use and interpretation of regression. Such capabilities are included in popular packages like MATLAB software and Excel. If your computer language has plotting capabilities, we recommend that you expand your program to include a plot of y versus x, showing both the data and the regression line. The inclusion of the capability will greatly enhance the utility of the program in problem-solving contexts.

cha9792x_ch17_456-489.indd Page 464 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 464 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 465

EXAMPLE 17.3 Linear Regression Using the Computer

Problem Statement. We can use software based on Fig. 17.6 to solve a hypothesis- testing problem associated with the falling parachutist discussed in Chap. 1. A theoreti- cal mathematical model for the velocity of the parachutist was given as the following [Eq. (1.10)]:

y(t) 5 gm

c (1 2 e(2cym)t)

where y 5 velocity (m/s), g 5 gravitational constant (9.8 m/s2), m 5 mass of the para- chutist equal to 68.1 kg, and c 5 drag coeffi cient of 12.5 kg/s. The model predicts the velocity of the parachutist as a function of time, as described in Example 1.1. An alternative empirical model for the velocity of the parachutist is given by

y(t) 5 gm

c a t

3.75 1 t b (E17.3.1)

Suppose that you would like to test and compare the adequacy of these two math- ematical models. This might be accomplished by measuring the actual velocity of the

SUB Regress(x, y, n, al, a0, syx, r2)

sumx 5 0: sumxy 5 0: st 5 0 sumy 5 0: sumx2 5 0: sr 5 0 DOFOR i 5 1, n sumx 5 sumx 1 xi sumy 5 sumy 1 yi sumxy 5 sumxy 1 xi*yi sumx2 5 sumx2 1 xi*xi END DO xm 5 sumx/n ym 5 sumy/n a1 5 (n*sumxy 2 sumx*sumy)y(n*sumx2 2 sumx*sumx) a0 5 ym 2 a1*xm DOFOR i 5 1, n st 5 st 1 (yi 2 ym)

2

sr 5 sr 1 (yi 2 a1*xi 2 a0) 2

END DO syx 5 (sr/(n 2 2))0.5

r2 5 (st 2 sr)/st

END Regress

FIGURE 17.6 Algorithm for linear regression.

cha9792x_ch17_456-489.indd Page 465 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 465 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

466 LEAST-SQUARES REGRESSION

parachutist at known values of time and comparing these results with the predicted ve- locities according to each model. Such an experimental-data-collection program was implemented, and the results are listed in column (a) of Table 17.2. Computed velocities for each model are listed in columns (b) and (c).

Solution. The adequacy of the models can be tested by plotting the model-calculated velocity versus the measured velocity. Linear regression can be used to calculate the slope and the intercept of the plot. This line will have a slope of 1, an intercept of 0, and an r2 5 1 if the model matches the data perfectly. A signifi cant deviation from these values can be used as an indication of the inadequacy of the model. Figure 17.7a and b are plots of the line and data for the regressions of columns (b) and (c), respectively, versus column (a). For the fi rst model [Eq. (1.10) as depicted in Fig. 17.7a],

ymodel 5 20.859 1 1.032ymeasure

and for the second model [Eq. (E17.3.1) as depicted in Fig. 17.7b],

ymodel 5 5.776 1 0.752ymeasure

These plots indicate that the linear regression between these data and each of the models is highly signifi cant. Both models match the data with a correlation coeffi cient of greater than 0.99. However, the model described by Eq. (1.10) conforms to our hypothesis test criteria much better than that described by Eq. (E17.3.1) because the slope and intercept are more nearly equal to 1 and 0. Thus, although each plot is well described by a straight line, Eq. (1.10) appears to be a better model than Eq. (E17.3.1).

TABLE 17.2 Measured and calculated velocities for the falling parachutist.

Measured v, Model-calculated v, Model-calculated v, m/s m/s [Eq. (1.10)] m/s [Eq. (E17.3.1)] Time, s (a) (b) (c)

1 10.00 8.953 11.240 2 16.30 16.405 18.570 3 23.00 22.607 23.729 4 27.50 27.769 27.556 5 31.00 32.065 30.509 6 35.60 35.641 32.855 7 39.00 38.617 34.766 8 41.50 41.095 36.351 9 42.90 43.156 37.687 10 45.00 44.872 38.829 11 46.00 46.301 39.816 12 45.50 47.490 40.678 13 46.00 48.479 41.437 14 49.00 49.303 42.110 15 50.00 49.988 42.712

cha9792x_ch17_456-489.indd Page 466 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 466 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 467

Model testing and selection are common and extremely important activities per- formed in all fi elds of engineering. The background material provided in this chapter, together with your software, should allow you to address many practical problems of this type.

55

30Y

5 30 X

55 5

(a)

55

30Y

5 30 X

55 5

(b) FIGURE 17.7 (a) Results using linear regression to compare predictions computed with the theoretical model [Eq. (1.10)] versus measured values. (b) Results using linear regression to compare predictions computed with the empirical model [Eq. (E17.3.1)] versus measured values.

There is one shortcoming with the analysis in Example 17.3. The example was un- ambiguous because the empirical model [Eq. (E17.3.1)] was clearly inferior to Eq. (1.10). Thus, the slope and intercept for the former were so much closer to the desired result of 1 and 0, that it was obvious which model was superior.

(a)

(b)

cha9792x_ch17_456-489.indd Page 467 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 467 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

468 LEAST-SQUARES REGRESSION

However, suppose that the slope were 0.85 and the intercept were 2. Obviously this would make the conclusion that the slope and intercept were 1 and 0 open to debate. Clearly, rather than relying on a subjective judgment, it would be preferable to base such a conclusion on a quantitative criterion. This can be done by computing confi dence intervals for the model parameters in the same way that we developed confi dence intervals for the mean in Sec. PT5.2.3. We will return to this topic at the end of this chapter.

17.1.5 Linearization of Nonlinear Relationships

Linear regression provides a powerful technique for fi tting a best line to data. However, it is predicated on the fact that the relationship between the dependent and independent variables is linear. This is not always the case, and the fi rst step in any regression analysis should be to plot and visually inspect the data to ascertain whether a linear model applies. For example, Fig. 17.8 shows some data that is obviously curvilinear. In some cases, techniques such as polynomial regression, which is described in Sec. 17.2, are appropriate. For others, transformations can be used to express the data in a form that is compatible with linear regression.

FIGURE 17.8 (a) Data that are ill-suited for linear least-squares regression. (b) Indication that a parabola is preferable.

y

x

(a)

y

x

(b)

cha9792x_ch17_456-489.indd Page 468 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 468 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 469

One example is the exponential model

y 5 a1e b1x (17.12)

where a1 and b1 are constants. This model is used in many fi elds of engineering to characterize quantities that increase (positive b1) or decrease (negative b1) at a rate that is directly proportional to their own magnitude. For example, population growth or ra- dioactive decay can exhibit such behavior. As depicted in Fig. 17.9a, the equation rep- resents a nonlinear relationship (for b1 ? 0) between y and x. Another example of a nonlinear model is the simple power equation

y 5 a2x b2 (17.13)

FIGURE 17.9 (a) The exponential equation, (b) the power equation, and (c) the saturation-growth-rate equation. Parts (d ), (e), and (f ) are linearized versions of these equations that result from simple transformations.

y

x

y = �1e �1x

(a)

L in

e a

ri za

ti o

n

y

x

y = �2x �2

(b)

L in

e a

ri za

ti o

n

y

x

(c)

L in

e a

ri za

ti o

n

y = �3 x

�3 + x

ln y

x

Slope = �1

Intercept = ln �1

(d)

log y

log x

(e)

1/y

1/x

( f )

Intercept = log �2

Intercept = 1/�3

Slope = �2 Slope = �3/�3

cha9792x_ch17_456-489.indd Page 469 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 469 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

470 LEAST-SQUARES REGRESSION

where a2 and b2 are constant coeffi cients. This model has wide applicability in all fi elds of engineering. As depicted in Fig. 17.9b, the equation (for b2 ? 0 or 1) is nonlinear. A third example of a nonlinear model is the saturation-growth-rate equation [recall Eq. (E17.3.1)]

y 5 a3 x

b3 1 x (17.14)

where a3 and b3 are constant coeffi cients. This model, which is particularly well-suited for characterizing population growth rate under limiting conditions, also represents a nonlinear relationship between y and x (Fig. 17.9c) that levels off, or “saturates,” as x increases. Nonlinear regression techniques are available to fi t these equations to experimental data directly. (Note that we will discuss nonlinear regression in Sec. 17.5.) However, a simpler alternative is to use mathematical manipulations to transform the equations into a linear form. Then, simple linear regression can be employed to fi t the equations to data. For example, Eq. (17.12) can be linearized by taking its natural logarithm to yield

ln y 5 ln a1 1 b1x ln e

But because ln e 5 1,

ln y 5 ln a1 1 b1x (17.15)

Thus, a plot of ln y versus x will yield a straight line with a slope of b1 and an intercept of ln a1 (Fig. 17.9d). Equation (17.13) is linearized by taking its base-10 logarithm to give

log y 5 b2 log x 1 log a2 (17.16)

Thus, a plot of log y versus log x will yield a straight line with a slope of b2 and an intercept of log a2 (Fig. 17.9e). Equation (17.14) is linearized by inverting it to give

1 y

5 b3

a3 1 x

1 1 a3

(17.17)

Thus, a plot of 1Yy versus lYx will be linear, with a slope of b3Ya3 and an intercept of 1Ya3 (Fig. 17.9f ). In their transformed forms, these models can use linear regression to evaluate the constant coeffi cients. They could then be transformed back to their original state and used for predictive purposes. Example 17.4 illustrates this procedure for Eq. (17.13). In addition, Sec. 20.1 provides an engineering example of the same sort of computation.

EXAMPLE 17.4 Linearization of a Power Equation

Problem Statement. Fit Eq. (17.13) to the data in Table 17.3 using a logarithmic transformation of the data.

Solution. Figure 17.10a is a plot of the original data in its untransformed state. Figure 17.10b shows the plot of the transformed data. A linear regression of the log-transformed data yields the result

log y 5 1.75 log x 2 0.300

cha9792x_ch17_456-489.indd Page 470 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 470 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.1 LINEAR REGRESSION 471

TABLE 17.3 Data to be fi t to the power equation.

x y log x log y

1 0.5 0 20.301 2 1.7 0.301 0.226 3 3.4 0.477 0.534 4 5.7 0.602 0.753 5 8.4 0.699 0.922

FIGURE 17.10 (a) Plot of untransformed data with the power equation that fi ts these data. (b) Plot of transformed data used to determine the coeffi cients of the power equation.

y

x50 0

5

(a)

log y

0.5

(b)

log x0.5

cha9792x_ch17_456-489.indd Page 471 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 471 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

472 LEAST-SQUARES REGRESSION

17.1.6 General Comments on Linear Regression

Before proceeding to curvilinear and multiple linear regression, we must emphasize the introductory nature of the foregoing material on linear regression. We have focused on the simple derivation and practical use of equations to fi t data. You should be cognizant of the fact that there are theoretical aspects of regression that are of practical importance but are beyond the scope of this book. For example, some statistical assumptions that are inherent in the linear least-squares procedures are

1. Each x has a fi xed value; it is not random and is known without error. 2. The y values are independent random variables and all have the same variance. 3. The y values for a given x must be normally distributed.

Such assumptions are relevant to the proper derivation and use of regression. For example, the fi rst assumption means that (1) the x values must be error-free and (2) the regression of y versus x is not the same as x versus y (try Prob. 17.4 at the end of the chapter). You are urged to consult other references such as Draper and Smith (1981) to appreciate aspects and nuances of regression that are beyond the scope of this book.

17.2 POLYNOMIAL REGRESSION

In Sec. 17.1, a procedure was developed to derive the equation of a straight line using the least-squares criterion. Some engineering data, although exhibiting a marked pattern such as seen in Fig. 17.8, is poorly represented by a straight line. For these cases, a curve would be better suited to fi t these data. As discussed in the previous section, one method to accomplish this objective is to use transformations. Another alternative is to fi t poly- nomials to the data using polynomial regression. The least-squares procedure can be readily extended to fi t the data to a higher-order polynomial. For example, suppose that we fi t a second-order polynomial or quadratic:

y 5 a0 1 a1x 1 a2x 2 1 e

For this case the sum of the squares of the residuals is [compare with Eq. (17.3)]

Sr 5 a n

i51 (yi 2 a0 2 a1xi 2 a2x2i )2 (17.18)

Following the procedure of the previous section, we take the derivative of Eq. (17.18) with respect to each of the unknown coeffi cients of the polynomial, as in

0Sr 0a0

5 22a (yi 2 a0 2 a1xi 2 a2x 2 i )

Thus, the intercept, log a2, equals 20.300, and therefore, by taking the antilogarithm, a2 5 10

20.3 5 0.5. The slope is b2 5 1.75. Consequently, the power equation is

y 5 0.5x1.75

This curve, as plotted in Fig. 17.10a, indicates a good fi t.

cha9792x_ch17_456-489.indd Page 472 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 472 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.2 POLYNOMIAL REGRESSION 473

0Sr 0a1

5 22a xi(yi 2 a0 2 a1xi 2 a2x 2 i )

0Sr 0a2

5 22a x 2 i (yi 2 a0 2 a1xi 2 a2x2i )

These equations can be set equal to zero and rearranged to develop the following set of normal equations:

(n)a0 1 (a xi)a1 1 (a x2i )a2 5 a yi (a xi)a0 1 (a x2i )a1 1 (a x3i )a2 5 a xiyi (17.19) (a x2i )a0 1 (a x3i )a1 1 (a x4i )a2 5 a x2i yi

where all summations are from i 5 1 through n. Note that the above three equations are linear and have three unknowns: a0, a1, and a2. The coeffi cients of the unknowns can be calculated directly from the observed data. For this case, we see that the problem of determining a least-squares second-order polynomial is equivalent to solving a system of three simultaneous linear equations. Techniques to solve such equations were discussed in Part Three. The two-dimensional case can be easily extended to an mth-order polynomial as

y 5 a0 1 a1x 1 a2x 2 1 p 1 amxm 1 e

The foregoing analysis can be easily extended to this more general case. Thus, we can recognize that determining the coeffi cients of an mth-order polynomial is equivalent to solving a system of m 1 1 simultaneous linear equations. For this case, the standard error is formulated as

sy/x 5 B

Sr n 2 (m 1 1)

(17.20)

This quantity is divided by n 2 (m 1 1) because (m 1 1) data-derived coeffi cients— a0, a1, . . . , am—were used to compute Sr; thus, we have lost m 1 1 degrees of free- dom. In addition to the standard error, a coeffi cient of determination can also be computed for polynomial regression with Eq. (17.10).

EXAMPLE 17.5 Polynomial Regression

Problem Statement. Fit a second-order polynomial to the data in the fi rst two columns of Table 17.4.

Solution. From the given data,

m 5 2 a xi 5 15 a x 4 i 5 979

n 5 6 a yi 5 152.6 a xiyi 5 585.6

x 5 2.5 a x 2 i 5 55 a x

2 i yi 5 2488.8

y 5 25.433 a x 3 i 5 225

cha9792x_ch17_456-489.indd Page 473 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 473 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

474 LEAST-SQUARES REGRESSION

Therefore, the simultaneous linear equations are

£ 6 15 55

15 55 225

55 225 979

§ • a0 a1 a2

¶ 5 • 152.6

585.6

2488.8

Solving these equations through a technique such as Gauss elimination gives a0 5 2.47857, a1 5 2.35929, and a2 5 1.86071. Therefore, the least-squares quadratic equation for this case is

y 5 2.47857 1 2.35929x 1 1.86071x2

The standard error of the estimate based on the regression polynomial is [Eq. (17.20)]

syyx 5 A

3.74657

6 2 3 5 1.12

TABLE 17.4 Computations for an error analysis of the quadratic least-squares fi t.

xi yi (yi 2 y )2 (yi 2 a0 2 a1xi 2 a2xi2)2

0 2.1 544.44 0.14332 1 7.7 314.47 1.00286 2 13.6 140.03 1.08158 3 27.2 3.12 0.80491 4 40.9 239.22 0.61951 5 61.1 1272.11 0.09439

S 152.6 2513.39 3.74657

FIGURE 17.11 Fit of a second-order polynomial.

y

x50

50 Least-squares

parabola

cha9792x_ch17_456-489.indd Page 474 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 474 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.2 POLYNOMIAL REGRESSION 475

The coeffi cient of determination is

r2 5 2513.39 2 3.74657

2513.39 5 0.99851

and the correlation coeffi cient is r 5 0.99925. These results indicate that 99.851 percent of the original uncertainty has been ex- plained by the model. This result supports the conclusion that the quadratic equation represents an excellent fi t, as is also evident from Fig. 17.11.

17.2.1 Algorithm for Polynomial Regression

An algorithm for polynomial regression is delineated in Fig. 17.12. Note that the primary task is the generation of the coeffi cients of the normal equations [Eq. (17.19)]. (Pseudocode for accomplishing this is presented in Fig. 17.13.) Then, techniques from Part Three can be applied to solve these simultaneous equations for the coeffi cients. A potential problem associated with implementing polynomial regression on the computer is that the normal equations tend to be ill-conditioned. This is particularly true for higher-order versions. For these cases, the computed coeffi cients may be highly susceptible to round-off error, and consequently, the results can be inaccurate. Among other things, this problem is related to the structure of the normal equations and to the fact that for higher-order polynomials the normal equations can have very large and very small coeffi cients. This is because the coeffi cients are summations of the data raised to powers. Although the strategies for mitigating round-off error discussed in Part Three, such as pivoting, can help to partially remedy this problem, a simpler alternative is to use a com- puter with higher precision. Fortunately, most practical problems are limited to lower-order polynomials for which round-off is usually negligible. In situations where higher-order versions are required, other alternatives are available for certain types of data. However, these techniques (such as orthogonal polynomials) are beyond the scope of this book. The reader should consult texts on regression, such as Draper and Smith (1981), for additional information regarding the problem and possible alternatives.

FIGURE 17.12 Algorithm for implementation of polynomial and multiple linear regression.

Step 1: Input order of polynomial to be fi t, m. Step 2: Input number of data points, n. Step 3: If n , m 1 1, print out an error message that regression is impossible and terminate

the process. If n $ m 1 1, continue. Step 4: Compute the elements of the normal equation in the form of an augmented matrix. Step 5: Solve the augmented matrix for the coeffi cients a0, a1, a2, . . . , am, using an

elimination method. Step 6: Print out the coeffi cients.

cha9792x_ch17_456-489.indd Page 475 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 475 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

476 LEAST-SQUARES REGRESSION

17.3 MULTIPLE LINEAR REGRESSION

A useful extension of linear regression is the case where y is a linear function of two or more independent variables. For example, y might be a linear function of x1 and x2, as in

y 5 a0 1 a1x1 1 a2x2 1 e

Such an equation is particularly useful when fi tting experimental data, where the variable being studied is often a function of two other variables. For this two-dimensional case, the regression “line” becomes a “plane” (Fig. 17.14).

DOFOR i 5 1, order 1 1 DOFOR j 5 1, i k 5 i 1 j 2 2 sum 5 0 DOFOR , 5 1, n sum 5 sum 1 x,

k

END DO ai,j 5 sum aj,i 5 sum END DO sum 5 0 DOFOR , 5 1, n sum 5 sum 1 y, ? x,

i21

END DO ai,order12 5 sum END DO

FIGURE 17.13 Pseudocode to assemble the elements of the normal equations for polynomial regression.

FIGURE 17.14 Graphical depiction of multiple linear regression where y is a linear function of x1 and x2.

y

x1

x2

cha9792x_ch17_456-489.indd Page 476 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 476 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.3 MULTIPLE LINEAR REGRESSION 477

As with the previous cases, the “best” values of the coeffi cients are determined by setting up the sum of the squares of the residuals,

Sr 5 a n

i51 (yi 2 a0 2 a1x1i 2 a2x2i)

2 (17.21)

and differentiating with respect to each of the unknown coeffi cients,

0Sr 0a0

5 22a (yi 2 a0 2 a1x1i 2 a2x2i)

0Sr 0a1

5 22a x1i (yi 2 a0 2 a1x1i 2 a2x2i)

0Sr 0a2

5 22a x2i (yi 2 a0 2 a1x1i 2 a2x2i)

The coeffi cients yielding the minimum sum of the squares of the residuals are obtained by setting the partial derivatives equal to zero and expressing the result in matrix form as

£ n gx1i gx2i gx1i gx

2 1i gx1i x2i

gx2i gx1ix2i gx 2 2i

§ 5 • a0 a1 a2

¶ 5 • gyi gx1i yi gx2i yi

¶ (17.22)

EXAMPLE 17.6 Multiple Linear Regression

Problem Statement. The following data were calculated from the equation y 5 5 1 4x1 2 3x2:

x1 x2 y

0 0 5 2 1 10 2.5 2 9 1 3 0 4 6 3 7 2 27

Use multiple linear regression to fi t these data.

Solution. The summations required to develop Eq. (17.22) are computed in Table 17.5. The result is

£ 6 16.5 14

16.5 76.25 48

14 48 54

§ • a0 a1 a2

¶ 5 • 54

243.5

100

which can be solved using a method such as Gauss elimination for

a0 5 5 a1 5 4 a2 5 23

which is consistent with the original equation from which these data were derived.

cha9792x_ch17_456-489.indd Page 477 30/10/13 8:20 PM f-512 cha9792x_ch17_456-489.indd Page 477 30/10/13 8:20 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

478 LEAST-SQUARES REGRESSION

The foregoing two-dimensional case can be easily extended to m dimensions, as in

y 5 a0 1 a1x1 1 a2x2 1 p 1 amxm 1 e

where the standard error is formulated as

syyx 5 B

Sr n 2 (m 1 1)

and the coeffi cient of determination is computed as in Eq. (17.10). An algorithm to set up the normal equations is listed in Fig. 17.15. Although there may be certain cases where a variable is linearly related to two or more other variables, multiple linear regression has additional utility in the derivation of power equations of the general form

y 5 a0 x a1 1 x

a2 2 p xamm

TABLE 17.5 Computations required to develop the normal equations for Example 17.6.

y x1 x2 x12 x22 x1x2 x1y x2y

5 0 0 0 0 0 0 0 10 2 1 4 1 2 20 10 9 2.5 2 6.25 4 5 22.5 18 0 1 3 1 9 3 0 0 3 4 6 16 36 24 12 18 27 7 2 49 4 14 189 54

S 54 16.5 14 76.25 54 48 243.5 100

DOFOR i 5 1, order 1 1 DOFOR j 5 1, i sum 5 0 DOFOR , 5 1, n sum 5 sum 1 xi21,, ? xj21,, END DO ai,j 5 sum aj,i 5 sum END DO sum 5 0 DOFOR , 5 1, n sum 5 sum 1 y, ? xi21,, END DO ai,order12 5 sum END DO

FIGURE 17.15 Pseudocode to assemble the elements of the normal equations for multiple regression. Note that aside from storing the independent variables in x1,i, x2,i, etc., 1’s must be stored in x0,i for this al- gorithm to work.

cha9792x_ch17_456-489.indd Page 478 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 478 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.4 GENERAL LINEAR LEAST SQUARES 479

Such equations are extremely useful when fi tting experimental data. To use multiple linear regression, the equation is transformed by taking its logarithm to yield

log y 5 log a0 1 a1 log x1 1 a2 log x2 1 p 1 am log xm This transformation is similar in spirit to the one used in Sec. 17.1.5 and Example 17.4 to fi t a power equation when y was a function of a single variable x. Section 20.4 provides an example of such an application for two independent variables.

17.4 GENERAL LINEAR LEAST SQUARES

To this point, we have focused on the mechanics of obtaining least-squares fi ts of some simple functions to data. Before turning to nonlinear regression, there are several issues that we would like to discuss to enrich your understanding of the preceding material.

17.4.1 General Matrix Formulation for Linear Least Squares

In the preceding pages, we have introduced three types of regression: simple linear, polynomial, and multiple linear. In fact, all three belong to the following general linear least-squares model:

y 5 a0 z0 1 a1z1 1 a2z2 1 p 1 am zm 1 e (17.23)

where z0, z1, . . . , zm are m 1 1 basis functions. It can easily be seen how simple and multiple linear regression fall within this model—that is, z0 5 1, z1 5 x1, z2 5 x2, . . . , zm 5 xm. Further, polynomial regression is also included if the basis functions are simple monomials as in z0 5 x

0 5 1, z1 5 x, z2 5 x 2, . . . , zm 5 x

m. Note that the terminology “linear” refers only to the model’s dependence on its parameters—that is, the a’s. As in the case of polynomial regression, the functions them- selves can be highly nonlinear. For example, the z’s can be sinusoids, as in

y 5 a0 1 a1 cos(vt) 1 a2 sin(vt)

Such a format is the basis of Fourier analysis described in Chap. 19. On the other hand, a simple-looking model like

f(x) 5 a0(1 2 e 2a1x)

is truly nonlinear because it cannot be manipulated into the format of Eq. (17.23). We will turn to such models at the end of this chapter. For the time being, Eq. (17.23) can be expressed in matrix notation as

{Y} 5 [Z]{A} 1 {E} (17.24)

where [Z] is a matrix of the calculated values of the basis functions at the measured values of the independent variables,

[Z] 5 Fz01 z11 p zm1z02 z12 p zm2. . . . . .

. . .

z0n z1n p zmn

V

cha9792x_ch17_456-489.indd Page 479 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 479 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

480 LEAST-SQUARES REGRESSION

where m is the number of variables in the model and n is the number of data points. Be- cause n $ m 1 1, you should recognize that most of the time, [Z] is not a square matrix. The column vector {Y} contains the observed values of the dependent variable

{Y}T 5 :y1 y2 p yn ; The column vector {A} contains the unknown coeffi cients

{A}T 5 :a0 a1 p am ; and the column vector {E} contains the residuals

{E}T 5 :e1 e2 p en ; As was done throughout this chapter, the sum of the squares of the residuals for this model can be defi ned as

Sr 5 a n

i51 ayi 2 a

m

j50 ajzjib

2

This quantity can be minimized by taking its partial derivative with respect to each of the coeffi cients and setting the resulting equation equal to zero. The outcome of this process is the normal equations that can be expressed concisely in matrix form as

3[Z]T[Z]4 {A} 5 5[Z]T{Y}6 (17.25) It can be shown that Eq. (17.25) is, in fact, equivalent to the normal equations developed previously for simple linear, polynomial, and multiple linear regression. Our primary motivation for the foregoing has been to illustrate the unity among the three approaches and to show how they can all be expressed simply in the same matrix notation. The matrix notation will also have relevance when we turn to nonlinear regres- sion in the last section of this chapter. From Eq. (PT3.6), recall that the matrix inverse can be employed to solve Eq. (17.25), as in

{A} 5 3[Z]T[Z]4215[Z]T{Y}6 (17.26) As we have learned in Part Three, this is an ineffi cient approach for solving a set of simultaneous equations. However, from a statistical perspective, there are a number of reasons why we might be interested in obtaining the inverse and examining its coeffi - cients. These reasons will be discussed next.

17.4.2 Statistical Aspects of Least-Squares Theory

In Sec. PT5.2.1, we reviewed a number of descriptive statistics that can be used to describe a sample. These included the arithmetic mean, the standard deviation, and the variance. Aside from yielding a solution for the regression coeffi cients, the matrix formula- tion of Eq. (17.26) provides estimates of their statistics. It can be shown (Draper and Smith, 1981) that the diagonal and off-diagonal terms of the matrix [[Z]T[Z]]21 give, respectively, the variances and the covariances1 of the a’s. If the diagonal elements of

1The covariance is a statistic that measures the dependency of one variable on another. Thus, cov(x, y) indicates the dependency of x and y. For example, cov(x, y) 5 0 would indicate that x and y are totally independent.

cha9792x_ch17_456-489.indd Page 480 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 480 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.4 GENERAL LINEAR LEAST SQUARES 481

[[Z]T[Z]]21 are designated as z21i,i ,

var(ai21) 5 z 21 i,i s

2 yyx (17.27)

and

cov(ai21, aj21) 5 z 21 i, j s

2 yyx (17.28)

These statistics have a number of important applications. For our present purposes, we will illustrate how they can be used to develop confi dence intervals for the intercept and slope. Using an approach similar to that in Sec. PT5.2.3, it can be shown that lower and upper bounds on the intercept can be formulated as (see Milton and Arnold, 2002, for details)

L 5 a0 2 tay2, n22 s(a0)  U 5 a0 1 tay2, n22 s(a0) (17.29)

where s(aj) 5 the standard error of coeffi cient aj 5 1var(aj). In a similar manner, lower and upper bounds on the slope can be formulated as

L 5 a1 2 tay2, n22 s(a1)  U 5 a1 1 tay2, n22 s(a1) (17.30)

The following example illustrates how these intervals can be used to make quantitative inferences related to linear regression.

EXAMPLE 17.7 Confi dence Intervals for Linear Regression

Problem Statement. In Example 17.3, we used regression to develop the following relationship between measurements and model predictions:

y 5 20.859 1 1.032x

where y 5 the model predictions and x 5 the measurements. We concluded that there was a good agreement between the two because the intercept was approximately equal to 0 and the slope approximately equal to 1. Recompute the regression but use the matrix approach to estimate standard errors for the parameters. Then employ these errors to develop confi dence intervals, and use these to make a probabilistic statement regarding the goodness of fi t.

Solution. These data can be written in matrix format for simple linear regression as:

[Z] 5 G 1 10

1 16.3

1 23

. .

. .

. .

1 50

W  {Y} 5 g 8.953

16.405

22.607

.

.

.

49.988

w Matrix transposition and multiplication can then be used to generate the normal equations as

3[Z]T[Z]4 {A} 5 5[Z]T{Y}6 c 15 548.3

548.3 22191.21 d ea0

a1 f 5 e 552.741

22421.43 f

cha9792x_ch17_456-489.indd Page 481 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 481 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

482 LEAST-SQUARES REGRESSION

Matrix inversion can be used to obtain the slope and intercept as

{A} 5 3[Z]T[Z]421 5[Z]T{Y}6 5 c 0.688414 20.01701

20.01701 0.000465 d e 552.741

22421.43 f 5 e20.85872

1.031592 f

Thus, the intercept and the slope are determined as a0 5 20.85872 and a1 5 1.031592, respectively. These values in turn can be used to compute the standard error of the estimate as syyx 5 0.863403. This value can be used along with the diagonal elements of the matrix inverse to calculate the standard errors of the coeffi cients,

s(a0) 5 2z 21 11 s

2 yyx 5 20.688414(0.863403)

2 5 0.716372

s(a1) 5 2z 21 22 s

2 yyx 5 20.000465(0.863403)

2 5 0.018625

The statistic, tay2,n21 needed for a 95% confi dence interval with n 2 2 5 15 2 2 5 13 degrees of freedom can be determined from a statistics table or using software. We used an Excel function, TINV, to come up with the proper value, as in

5 TINV(0.05, 13)

which yielded a value of 2.160368. Equations (17.29) and (17.30) can then be used to compute the confi dence intervals as

a0 5 20.85872 ; 2.160368(0.716372) 5 20.85872 ; 1.547627 5 [22.40634, 0.688912]

a1 5 1.031592 ; 2.160368(0.018625) 5 1.031592 ; 0.040237 5 [0.991355, 1.071828]

Notice that the desired values (0 for intercept and slope and 1 for the intercept) fall within the intervals. On the basis of this analysis we could make the following statement regarding the slope: We have strong grounds for believing that the slope of the true regres- sion line lies within the interval from 0.991355 to 1.071828. Because 1 falls within this interval, we also have strong grounds for believing that the result supports the agreement between the measurements and the model. Because zero falls within the intercept interval, a similar statement can be made regarding the intercept.

As mentioned previously in Sec. 17.2.1, the normal equations are notoriously ill- conditioned. Hence, if solved with conventional techniques such as LU decomposition, the computed coeffi cients can be highly susceptible to round-off error. As a conse- quence, more sophisticated orthogonalization algorithms, such as QR factorization, are available to circumvent the problem. Because these techniques are beyond the scope of this book, the reader should consult texts on regression, such as Draper and Smith (1981), for additional information regarding the problem and possible alternatives. Moler (2004) also provides a nice discussion of the topic with emphasis on the nu- merical methods. The foregoing is a limited introduction to the rich topic of statistical inference and its relationship to regression. There are many subleties that are beyond the scope of this

cha9792x_ch17_456-489.indd Page 482 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 482 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.5 NONLINEAR REGRESSION 483

book. Our primary motivation has been to illustrate the power of the matrix approach to general linear least squares. In addition, it should be noted that software packages such as Excel, MATLAB, and Mathcad can generate least-squares regression fi ts along with information relevant to inferential statistics. We will explore some of these capabilities when we describe these packages at the end of Chap. 19.

17.5 NONLINEAR REGRESSION

There are many cases in engineering where nonlinear models must be fi t to data. In the present context, these models are defi ned as those that have a nonlinear dependence on their parameters. For example,

f(x) 5 a0(1 2 e 2a1x) 1 e (17.31)

This equation cannot be manipulated so that it conforms to the general form of Eq. (17.23). As with linear least squares, nonlinear regression is based on determining the values of the parameters that minimize the sum of the squares of the residuals. However, for the nonlinear case, the solution must proceed in an iterative fashion. The Gauss-Newton method is one algorithm for minimizing the sum of the squares of the residuals between data and nonlinear equations. The key concept underlying the technique is that a Taylor series expansion is used to express the original nonlinear equa- tion in an approximate, linear form. Then, least-squares theory can be used to obtain new estimates of the parameters that move in the direction of minimizing the residual. To illustrate how this is done, fi rst the relationship between the nonlinear equation and the data can be expressed generally as

yi 5 f(xi; a0, a1, p , am) 1 ei where yi 5 a measured value of the dependent variable, f(xi ; a0, a1, p , am) 5 the equa- tion that is a function of the independent variable xi and a nonlinear function of the parameters a0, a1, p , am, and ei 5 a random error. For convenience, this model can be expressed in abbreviated form by omitting the parameters,

yi 5 f(xi) 1 ei (17.32)

The nonlinear model can be expanded in a Taylor series around the parameter values and curtailed after the fi rst derivative. For example, for a two-parameter case,

f(xi)j11 5 f(xi)j 1 0f(xi)j

0a0 ¢a0 1

0f(xi)j 0a1

¢a1 (17.33)

where j 5 the initial guess, j 1 1 5 the prediction, Da0 5 a0, j11 2 a0, j, and Da1 5 a1, j11 2 a1, j. Thus, we have linearized the original model with respect to the parameters. Equation (17.33) can be substituted into Eq. (17.32) to yield

yi 2 f(xi)j 5 0f(xi)j

0a0 ¢a0 1

0f(xi)j 0a1

¢a1 1 ei

or in matrix form [compare with Eq. (17.24)],

{D} 5 [Zj]{¢A} 1 {E} (17.34)

cha9792x_ch17_456-489.indd Page 483 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 483 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

484 LEAST-SQUARES REGRESSION

where [Zj] is the matrix of partial derivatives of the function evaluated at the initial guess j,

[Zj] 5 F0f1y0a0 0f1y0a10f2y0a0 0f2y0a1. .. . . .

0fny0a0 0fny0a1

V where n 5 the number of data points and 0fiy0ak 5 the partial derivative of the function with respect to the kth parameter evaluated at the ith data point. The vector {D} contains the differences between the measurements and the function values,

{D} 5 fy1 2 f(x1)y2 2 f(x2). .

.

yn 2 f(xn)

v and the vector {DA} contains the changes in the parameter values,

{¢A} 5 f¢a0¢a1. .

.

¢am

v Applying linear least-squares theory to Eq. (17.34) results in the following normal equa- tions [recall Eq. (17.25)]:

3[Zj]T[Zj]4 {¢A} 5 5[Zj]T{D}6 (17.35) Thus, the approach consists of solving Eq. (17.35) for {DA}, which can be employed to compute improved values for the parameters, as in

a0, j11 5 a0, j 1 ¢a0

and

a1, j11 5 a1, j 1 ¢a1

This procedure is repeated until the solution converges—that is, until

Zea Z k 5 ` ak, j11 2 ak, jak, j11 ` 100% (17.36) falls below an acceptable stopping criterion.

cha9792x_ch17_456-489.indd Page 484 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 484 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

17.5 NONLINEAR REGRESSION 485

EXAMPLE 17.8 Gauss-Newton Method

Problem Statement. Fit the function f(x; a0, a1) 5 a0(1 2 e2a1x) to the data:

x 0.25 0.75 1.25 1.75 2.25

y 0.28 0.57 0.68 0.74 0.79

Use initial guesses of a0 5 1.0 and a1 5 1.0 for the parameters. Note that for these guesses, the initial sum of the squares of the residuals is 0.0248.

Solution. The partial derivatives of the function with respect to the parameters are

0f 0a0

5 1 2 e2a1x (E17.8.1)

and

0f 0a1

5 a0xe 2a1x (E17.8.2)

Equations (E17.8.1) and (E17.8.2) can be used to evaluate the matrix

[Z0] 5 E0.2212 0.19470.5276 0.35430.7135 0.3581 0.8262 0.3041

0.8946 0.2371

U This matrix multiplied by its transpose results in

[Z0] T[Z0] 5 c2.3193 0.94890.9489 0.4404 d

which in turn can be inverted to yield

3[Z0]T[Z0]421 5 c 3.6397 27.8421 27.8421 19.1678

d The vector {D} consists of the differences between the measurements and the model predictions,

{D} 5 e0.28 2 0.22120.57 2 0.52760.68 2 0.7135 0.74 2 0.8262

0.79 2 0.8946

u 5 e 0.05880.042420.0335 20.0862

20.1046

u It is multiplied by [Z0]

T to give

[Z0] T{D} 5 c20.1533

20.0365 d

cha9792x_ch17_456-489.indd Page 485 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 485 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

486 LEAST-SQUARES REGRESSION

The vector {DA} is then calculated by solving Eq. (17.35) for

¢A 5 e20.2714 0.5019

f which can be added to the initial parameter guesses to yield

ea0 a1 f 5 e1.0

1.0 f 1 e20.2714

0.5019 f 5 e0.7286

1.5019 f

Thus, the improved estimates of the parameters are a0 5 0.7286 and a1 5 1.5019. The new parameters result in a sum of the squares of the residuals equal to 0.0242. Equation (17.36) can be used to compute e0 and e1 equal to 37 and 33 percent, respectively. The computation would then be repeated until these values fell below the prescribed stopping criterion. The fi nal result is a0 5 0.79186 and a1 5 1.6751. These coeffi cients give a sum of the squares of the residuals of 0.000662.

A potential problem with the Gauss-Newton method as developed to this point is that the partial derivatives of the function may be diffi cult to evaluate. Consequently, many computer programs use difference equations to approximate the partial derivatives. One method is

0fi 0ak

> f(xi; a0, p , ak 1 dak, p , am) 2 f(xi; a0, p , ak, p , am)

dak (17.37)

where d 5 a small fractional perturbation. The Gauss-Newton method has a number of other possible shortcomings:

1. It may converge slowly. 2. It may oscillate widely, that is, continually change directions. 3. It may not converge at all.

Modifi cations of the method (Booth and Peterson, 1958; Hartley, 1961) have been de- veloped to remedy the shortcomings. In addition, although there are several approaches expressly designed for regres- sion, a more general approach is to use nonlinear optimization routines as described in Part Four. To do this, a guess for the parameters is made, and the sum of the squares of the residuals is computed. For example, for Eq. (17.31) it would be com- puted as

Sr 5 a n

i51 [yi 2 a0(1 2 e

2a1xi)]2 (17.38)

Then, the parameters would be adjusted systematically to minimize Sr using search tech- niques of the type described previously in Chap. 14. We will illustrate how this is done when we describe software applications at the end of Chap. 19.

cha9792x_ch17_456-489.indd Page 486 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 486 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 487

PROBLEMS

17.1 Given these data

8.8 9.5 9.8 9.4 10.0 9.4 10.1 9.2 11.3 9.4

10.0 10.4 7.9 10.4 9.8 9.8 9.5 8.9 8.8 10.6

10.1 9.5 9.6 10.2 8.9

Determine (a) the mean, (b) the standard deviation, (c) the vari- ance, (d) the coeffi cient of variation, and (e) the 95% confi dence interval for the mean. (f) construct a histogram using a range from 7.5 to 11.5 with intervals of 0.5. 17.2 Given these data

29.65 28.55 28.65 30.15 29.35 29.75 29.25 30.65 28.15 29.85 29.05 30.25 30.85 28.75 29.65 30.45 29.15 30.45 33.65 29.35 29.75 31.25 29.45 30.15 29.65 30.55 29.65 29.25

Determine (a) the mean, (b) the standard deviation, (c) the vari- ance, (d) the coeffi cient of variation, and (e) the 90% confi dence interval for the mean. (f) Construct a histogram. Use a range from 28 to 34 with increments of 0.4. (g) Assuming that the distribution is normal and that your estimate of the standard deviation is valid, compute the range (that is, the lower and the upper values) that encompasses 68% of the readings. Determine whether this is a valid estimate for the data in this problem. 17.3 Use least-squares regression to fi t a straight line to

x 0 2 4 6 9 11 12 15 17 19

y 5 6 7 6 9 8 7 10 12 12

Along with the slope and intercept, compute the standard error of the estimate and the correlation coeffi cient. Plot the data and the regression line. Then repeat the problem, but regress x versus y— that is, switch the variables. Interpret your results. 17.4 Use least-squares regression to fi t a straight line to

x 6 7 11 15 17 21 23 29 29 37 39

y 29 21 29 14 21 15 7 7 13 0 3

Along with the slope and the intercept, compute the standard error of the estimate and the correlation coeffi cient. Plot the data and the re- gression line. If someone made an additional measurement of x 5 10, y 5 10, would you suspect, based on a visual assessment and the standard error, that the measurement was valid or faulty? Justify your conclusion. 17.5 Using the same approach as was employed to derive Eqs. (17.15) and (17.16), derive the least-squares fi t of the following model:

y 5 a1x 1 e

That is, determine the slope that results in the least-squares fi t for a straight line with a zero intercept. Fit the following data with this model and display the result graphically:

x 2 4 6 7 10 11 14 17 20

y 1 2 5 2 8 7 6 9 12

17.6 Use least-squares regression to fi t a straight line to

x 1 2 3 4 5 6 7 8 9

y 1 1.5 2 3 4 5 8 10 13

(a) Along with the slope and intercept, compute the standard error of the estimate and the correlation coeffi cient. Plot the data and the straight line. Assess the fi t.

(b) Recompute (a), but use polynomial regression to fi t a parabola to the data. Compare the results with those of (a).

17.7 Fit the following data with (a) a saturation-growth-rate model, (b) a power equation, and (c) a parabola. In each case, plot the data and the equation.

x 0.75 2 3 4 6 8 8.5

y 1.2 1.95 2 2.4 2.4 2.7 2.6

17.8 Fit the following data with the power model (y 5 axb). Use the resulting power equation to predict y at x 5 9:

x 2.5 3.5 5 6 7.5 10 12.5 15 17.5 20

y 13 11 8.5 8.2 7 6.2 5.2 4.8 4.6 4.3

17.9 Fit an exponential model to

x 0.4 0.8 1.2 1.6 2 2.3

y 800 975 1500 1950 2900 3600

Plot the data and the equation on both standard and semi-logarithmic graph paper. 17.10 Rather than using the base-e exponential model (Eq. 17.22), a common alternative is to use a base-10 model,

y 5 a510 b5x

When used for curve fi tting, this equation yields identical results to the base-e version, but the value of the exponent parameter (b5) will differ from that estimated with Eq. 17.22 (b1). Use the base-10 version to solve Prob. 17.9. In addition, develop a formulation to relate b1 to b5. 17.11 Beyond the examples in Fig. 17.10, there are other models that can be linearized using transformations. For example,

y 5 a4xe b4x

cha9792x_ch17_456-489.indd Page 487 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 487 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

488 LEAST-SQUARES REGRESSION

Determine the coeffi cients by setting up and solving Eq. (17.25). 17.16 Given these data

x 5 10 15 20 25 30 35 40 45 50

y 17 24 31 33 37 37 40 40 42 41

use least-squares regression to fi t (a) a straight line, (b) a power equation, (c) a saturation-growth-rate equation, and (d) a parabola. Plot the data along with all the curves. Is any one of the curves superior? If so, justify. 17.17 Fit a cubic equation to the following data:

x 3 4 5 7 8 9 11 12

y 1.6 3.6 4.4 3.4 2.2 2.8 3.8 4.6

Along with the coeffi cients, determine r2 and syyx. 17.18 Use multiple linear regression to fi t

x1 0 1 1 2 2 3 3 4 4

x2 0 1 2 1 2 1 2 1 2

y 15.1 17.9 12.7 25.6 20.5 35.1 29.7 45.4 40.2

Compute the coeffi cients, the standard error of the estimate, and the correlation coeffi cient. 17.19 Use multiple linear regression to fi t

x1 0 0 1 2 0 1 2 2 1

x2 0 2 2 4 4 6 6 2 1

y 14 21 11 12 23 23 14 6 11

Compute the coeffi cients, the standard error of the estimate, and the correlation coeffi cient. 17.20 Use nonlinear regression to fi t a parabola to the following data:

x 0.2 0.5 0.8 1.2 1.7 2 2.3

y 500 700 1000 1200 2200 2650 3750

17.21 Use nonlinear regression to fi t a saturation-growth-rate equation to the data in Prob. 17.16. 17.22 Recompute the regression fi ts from Probs. (a) 17.3 and (b) 17.17, using the matrix approach. Estimate the standard errors and develop 90% confi dence intervals for the coeffi cients. 17.23 Develop, debug, and test a program in either a high-level language or macro language of your choice to implement linear regression. Among other things: (a) include statements to docu- ment the code, and (b) determine the standard error and the coeffi - cient of determination. 17.24 A material is tested for cyclic fatigue failure whereby a stress, in MPa, is applied to the material and the number of cycles needed to cause failure is measured. The results are in the table below. When a log-log plot of stress versus cycles is generated, the

Linearize this model and use it to estimate a4 and b4 based on the following data. Develop a plot of your fi t along with the data.

x 0.1 0.2 0.4 0.6 0.9 1.3 1.5 1.7 1.8

y 0.75 1.25 1.45 1.25 0.85 0.55 0.35 0.28 0.18

17.12 An investigator has reported the data tabulated below for an experiment to determine the growth rate of bacteria k (per d), as a function of oxygen concentration c (mg/L). It is known that such data can be modeled by the following equation:

k 5 kmaxc

2

cs 1 c 2

where cs and kmax are parameters. Use a transformation to linearize this equation. Then use linear regression to estimate cs and kmax and predict the growth rate at c 5 2 mg/L.

c 0.5 0.8 1.5 2.5 4

k 1.1 2.4 5.3 7.6 8.9

17.13 An investigator has reported the data tabulated below. It is known that such data can be modeled by the following equation

x 5 e(y2b)ya

where a and b are parameters. Use a transformation to linearize this equation and then employ linear regression to determine a and b. Based on your analysis predict y at x 5 2.6.

x 1 2 3 4 5

y 0.5 2 2.9 3.5 4

17.14 It is known that the data tabulated below can be modeled by the following equation

y 5 aa 1 1x b1x

 b2

Use a transformation to linearize this equation and then employ linear regression to determine the parameters a and b. Based on your analysis predict y at x 5 1.6.

x 0.5 1 2 3 4

y 10.4 5.8 3.3 2.4 2

17.15 The following data are provided

x 1 2 3 4 5

y 2.2 2.8 3.6 4.5 5.5

You want to use least-squares regression to fi t these data with the following model,

y 5 a 1 bx 1 c

x

cha9792x_ch17_456-489.indd Page 488 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 488 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 489

at which the concentration will reach 200 CFUy100 mL. Note that your choice of model should be consistent with the fact that nega- tive concentrations are impossible and that the bacteria concentra- tion always decreases with time. 17.28 An object is suspended in a wind tunnel and the force mea- sured for various levels of wind velocity. The results are tabulated below.

v, m/s 10 20 30 40 50 60 70 80

F, N 25 70 380 550 610 1220 830 1450

Use least-squares regression to fi t these data with (a) a straight line, (b) a power equation based on log transformations, and (c) a power model based on nonlinear regression. Display the results graphically. 17.29 Fit a power model to the data from Prob. 17.28, but use natural logarithms to perform the transformations. 17.30 Derive the least-squares fi t of the following model:

y 5 a1x 1 a2x 2 1 e

That is, determine the coeffi cients that results in the least-squares fi t for a second-order polynomial with a zero intercept. Test the ap- proach by using it to fi t the data from Prob. 17.28. 17.31 In Prob. 17.11 we used transformations to linearize and fi t the following model:

y 5 a4xe b4x

Use nonlinear regression to estimate a4 and b4 based on the follow- ing data. Develop a plot of your fi t along with the data.

x 0.1 0.2 0.4 0.6 0.9 1.3 1.5 1.7 1.8

y 0.75 1.25 1.45 1.25 0.85 0.55 0.35 0.28 0.18

data trend shows a linear relationship. Use least-squares regression to determine a best-fi t equation for these data.

N, cycles 1 10 100 1000 10,000 100,000 1,000,000

Stress, MPa 1100 1000 925 800 625 550 420

17.25 The following data show the relationship between the vis- cosity of SAE 70 oil and temperature. After taking the log of the data, use linear regression to fi nd the equation of the line that best fi ts the data and the r2 value.

Temperature, 8C 26.67 93.33 148.89 315.56

Viscosity, m, N ? s/m2 1.35 0.085 0.012 0.00075

17.26 The data below represents the bacterial growth in a liquid culture over a number of days.

Day 0 4 8 12 16 20

Amount 3 106 67 84 98 125 149 185

Find a best-fi t equation to the data trend. Try several possibilities— linear, parabolic, and exponential. Use the software package of your choice to fi nd the best equation to predict the amount of bac- teria after 40 days. 17.27 The concentration of E. coli bacteria in a swimming area is monitored after a storm:

t (hr) 4 8 12 16 20 24

c (CFUy100 mL) 1600 1320 1000 890 650 560

The time is measured in hours following the end of the storm and the unit CFU is a “colony forming unit.” Use these data to estimate (a) the concentration at the end of the storm (t 5 0) and (b) the time

cha9792x_ch17_456-489.indd Page 489 30/10/13 8:21 PM f-512 cha9792x_ch17_456-489.indd Page 489 30/10/13 8:21 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18 C H A P T E R 18

490

Interpolation

You will frequently have occasion to estimate intermediate values between precise data points. The most common method used for this purpose is polynomial interpolation. Recall that the general formula for an nth-order polynomial is

f(x) 5 a0 1 a1x 1 a2x 2 1 p 1 anxn (18.1)

For n 1 1 data points, there is one and only one polynomial of order n that passes through all the points. For example, there is only one straight line (that is, a fi rst-order polynomial) that connects two points (Fig. 18.1a). Similarly, only one parabola connects a set of three points (Fig. 18.lb). Polynomial interpolation consists of determining the unique nth-order polynomial that fi ts n 1 1 data points. This polynomial then provides a formula to compute intermediate values. Although there is one and only one nth-order polynomial that fi ts n 1 1 points, there are a variety of mathematical formats in which this polynomial can be expressed. In this chapter, we will describe two alternatives that are well-suited for computer implementa- tion: the Newton and the Lagrange polynomials.

FIGURE 18.1 Examples of interpolating polynomials: (a) fi rst-order (linear) connecting two points, (b) second- order (quadratic or parabolic) connecting three points, and (c) third-order (cubic) connecting four points.

(a) (b) (c)

cha9792x_ch18_490-525.indd Page 490 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 490 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 491

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS

As stated above, there are a variety of alternative forms for expressing an interpolating polynomial. Newton’s divided-difference interpolating polynomial is among the most popular and useful forms. Before presenting the general equation, we will introduce the fi rst- and second-order versions because of their simple visual interpretation.

18.1.1 Linear Interpolation

The simplest form of interpolation is to connect two data points with a straight line. This tech- nique, called linear interpolation, is depicted graphically in Fig. 18.2. Using similar triangles,

f1(x) 2 f(x0) x 2 x0

5 f(x1) 2 f(x0)

x1 2 x0

which can be rearranged to yield

f1(x) 5 f(x0) 1 f(x1) 2 f(x0)

x1 2 x0 (x 2 x0) (18.2)

which is a linear-interpolation formula. The notation f1(x) designates that this is a fi rst- order interpolating polynomial. Notice that besides representing the slope of the line connecting the points, the term [ f(x1) 2 f(x0)]y(x1 2 x0) is a fi nite-divided-difference

FIGURE 18.2 Graphical depiction of linear interpolation. The shaded areas indicate the similar triangles used to derive the linear-interpolation formula [Eq. (18.2)].

f (x)

xx1xx0

f (x1)

f (x0)

f1(x)

cha9792x_ch18_490-525.indd Page 491 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 491 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

492 INTERPOLATION

approximation of the fi rst derivative [recall Eq. (4.17)]. In general, the smaller the inter- val between the data points, the better the approximation. This is due to the fact that, as the interval decreases, a continuous function will be better approximated by a straight line. This characteristic is demonstrated in the following example.

EXAMPLE 18.1 Linear Interpolation

Problem Statement. Estimate the natural logarithm of 2 using linear interpolation. First, perform the computation by interpolating between ln 1 5 0 and ln 6 5 1.791759. Then, repeat the procedure, but use a smaller interval from ln 1 to ln 4 (1.386294). Note that the true value of ln 2 is 0.6931472.

Solution. We use Eq. (18.2) and a linear interpolation for ln(2) from x0 5 1 to x1 5  6 to give

f1(2) 5 0 1 1.791759 2 0

6 2 1 (2 2 1) 5 0.3583519

which represents an error of �t 5 48.3%. Using the smaller interval from x0 5 1 to x1 5 4 yields

f1(2) 5 0 1 1.386294 2 0

4 2 1 (2 2 1) 5 0.4620981

Thus, using the shorter interval reduces the percent relative error to �t 5 33.3%. Both interpolations are shown in Fig. 18.3, along with the true function.

FIGURE 18.3 Two linear interpolations to estimate ln 2. Note how the smaller interval provides a better estimate.

f (x)

f (x) = ln x

f1(x)

True value

Linear estimates

x50

2

0

1

cha9792x_ch18_490-525.indd Page 492 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 492 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 493

18.1.2 Quadratic Interpolation

The error in Example 18.1 resulted from our approximating a curve with a straight line. Consequently, a strategy for improving the estimate is to introduce some curvature into the line connecting the points. If three data points are available, this can be accomplished with a second-order polynomial (also called a quadratic polynomial or a parabola). A particularly convenient form for this purpose is

f2(x) 5 b0 1 b1(x 2 x0) 1 b2(x 2 x0) (x 2 x1) (18.3)

Note that although Eq. (18.3) might seem to differ from the general polynomial [Eq. (18.1)], the two equations are equivalent. This can be shown by multiplying the terms in Eq.  (18.3) to yield

f2(x) 5 b0 1 b1x 2 b1x0 1 b2x 2 1 b2x0x1 2 b2xx0 2 b2xx1

or, collecting terms,

f2(x) 5 a0 1 a1x 1 a2x 2

where

a0 5 b0 2 b1x0 1 b2x0x1 a1 5 b1 2 b2x0 2 b2x1 a2 5 b2

Thus, Eqs. (18.1) and (18.3) are alternative, equivalent formulations of the unique second- order polynomial joining the three points. A simple procedure can be used to determine the values of the coeffi cients. For b0, Eq. (18.3) with x 5 x0 can be used to compute

b0 5 f(x0) (18.4)

Equation (18.4) can be substituted into Eq. (18.3), which can be evaluated at x 5 x1 for

b1 5 f(x1) 2 f(x0)

x1 2 x0 (18.5)

Finally, Eqs. (18.4) and (18.5) can be substituted into Eq. (18.3), which can be evaluated at x 5 x2 and solved (after some algebraic manipulations) for

b2 5

f(x2) 2 f(x1) x2 2 x1

2 f(x1) 2 f(x0)

x1 2 x0 x2 2 x0

(18.6)

Notice that, as was the case with linear interpolation, b1 still represents the slope of the line connecting points x0 and x1. Thus, the fi rst two terms of Eq. (18.3) are equivalent to linear interpolation from x0 to x1, as specifi ed previously in Eq. (18.2). The last term, b2(x 2 x0)(x 2 x1), introduces the second-order curvature into the formula. Before illustrating how to use Eq. (18.3), we should examine the form of the coef- fi cient b2. It is very similar to the fi nite-divided-difference approximation of the second derivative introduced previously in Eq. (4.24). Thus, Eq. (18.3) is beginning to manifest a structure that is very similar to the Taylor series expansion. This observation will be

cha9792x_ch18_490-525.indd Page 493 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 493 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

494 INTERPOLATION

explored further when we relate Newton’s interpolating polynomials to the Taylor series in Sec. 18.1.4. But fi rst, we will do an example that shows how Eq. (18.3) is used to interpolate among three points.

EXAMPLE 18.2 Quadratic Interpolation

Problem Statement. Fit a second-order polynomial to the three points used in Example 18.1:

x0 5 1  f(x0) 5 0

x1 5 4  f(x1) 5 1.386294

x2 5 6  f(x2) 5 1.791759

Use the polynomial to evaluate ln 2.

Solution. Applying Eq. (18.4) yields

b0 5 0

Equation (18.5) yields

b1 5 1.386294 2 0

4 2 1 5 0.4620981

and Eq. (18.6) gives

b2 5

1.791759 2 1.386294

6 2 4 2 0.4620981

6 2 1 5 20.0518731

FIGURE 18.4 The use of quadratic interpolation to estimate ln 2. The linear interpolation from x 5 1 to 4 is also included for comparison.

f (x)

f (x) = ln x

f2(x)

True value

Linear estimate

Quadratic estimate

x50

2

0

1

cha9792x_ch18_490-525.indd Page 494 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 494 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 495

Substituting these values into Eq. (18.3) yields the quadratic formula

f2(x) 5 0 1 0.4620981(x 2 1) 2 0.0518731(x 2 1)(x 2 4)

which can be evaluated at x 5 2 for

f2(2) 5 0.5658444

which represents a relative error of �t 5 18.4%. Thus, the curvature introduced by the quadratic formula (Fig. 18.4) improves the interpolation compared with the result obtained using straight lines in Example 18.1 and Fig. 18.3.

18.1.3 General Form of Newton’s Interpolating Polynomials

The preceding analysis can be generalized to fi t an nth-order polynomial to n 1 1 data points. The nth-order polynomial is

fn(x) 5 b0 1 b1(x 2 x0) 1 p 1 bn(x 2 x0) (x 2 x1) p (x 2 xn21) (18.7)

As was done previously with the linear and quadratic interpolations, data points can be used to evaluate the coeffi cients b0, b1, . . . , bn. For an nth-order polynomial, n 1 1 data points are required: [x0, f(x0)], [x1, f(x1)], . . . , [xn, f(xn)]. We use these data points and the following equations to evaluate the coeffi cients:

b0 5 f(x0) (18.8)

b1 5 f [x1, x0] (18.9)

b2 5 f [x2, x1, x0] (18.10)

.

.

.

bn 5 f [xn, xn21, p , x1, x0] (18.11)

where the bracketed function evaluations are fi nite divided differences. For example, the fi rst fi nite divided difference is represented generally as

f [xi, xj] 5 f(xi) 2 f(xj)

xi 2 xj (18.12)

The second fi nite divided difference, which represents the difference of two fi rst divided differences, is expressed generally as

f [xi, xj, xk] 5 f [xi, xj] 2 f [xj, xk]

xi 2 xk (18.13)

Similarly, the nth fi nite divided difference is

f [xn, xn21, p , x1, x0] 5 f [xn, xn21, p , x1] 2 f [xn21, xn22, p , x0]

xn 2 x0 (18.14)

cha9792x_ch18_490-525.indd Page 495 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 495 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

496 INTERPOLATION

These differences can be used to evaluate the coeffi cients in Eqs. (18.8) through (18.11), which can then be substituted into Eq. (18.7) to yield the interpolating polynomial

fn(x) 5 f(x0) 1 (x 2 x0) f [x1, x0] 1 (x 2 x0) (x 2 x1) f [x2, x1, x0]

1 p 1 (x 2 x0) (x 2 x1) p (x 2 xn21) f [xn, xn21, p , x0] (18.15)

which is called Newton’s divided-difference interpolating polynomial. It should be noted that it is not necessary that the data points used in Eq. (18.15) be equally spaced or that the abscissa values necessarily be in ascending order, as illustrated in the following example. Also, notice how Eqs. (18.12) through (18.14) are recursive—that is, higher- order differences are computed by taking differences of lower-order differences (Fig. 18.5). This property will be exploited when we develop an effi cient computer program in Sec. 18.1.5 to implement the method.

EXAMPLE 18.3 Newton’s Divided-Difference Interpolating Polynomials

Problem Statement. In Example 18.2, data points at x0 5 1, x1 5 4, and x2 5 6 were used to estimate ln 2 with a parabola. Now, adding a fourth point [x3 5 5; f(x3) 5 1.609438], estimate ln 2 with a third-order Newton’s interpolating polynomial.

Solution. The third-order polynomial, Eq. (18.7) with n 5 3, is

f3(x) 5 b0 1 b1(x 2 x0) 1 b2(x 2 x0) (x 2 x1) 1 b3(x 2 x0) (x 2 x1) (x 2 x2)

The fi rst divided differences for the problem are [Eq. (18.12)]

f [x1, x0] 5 1.386294 2 0

4 2 1 5 0.4620981

f [x2, x1] 5 1.791759 2 1.386294

6 2 4 5 0.2027326

f [x3, x2] 5 1.609438 2 1.791759

5 2 6 5 0.1823216

FIGURE 18.5 Graphical depiction of the recursive nature of fi nite divided differences.

i xi f(xi) First Second Third

0 x0 f(x0) f [x1, x0] f [x2, x1, x0] f [x3, x2, x1, x0] 1 x1 f(x1) f [x2, x1] f [x3, x2, x1] 2 x2 f(x2) f [x3, x2] 3 x3 f(x3)

cha9792x_ch18_490-525.indd Page 496 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 496 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 497

The second divided differences are [Eq. (18.13)]

f [x2, x1, x0] 5 0.2027326 2 0.4620981

6 2 1 5 20.05187311

f [x3, x2, x1] 5 0.1823216 2 0.2027326

5 2 4 5 20.02041100

The third divided difference is [Eq. (18.14) with n 5 3]

f [x3, x2, x1, x0] 5 20.02041100 2 (20.05187311)

5 2 1 5 0.007865529

The results for f [x1, x0], f [x2, x1, x0], and f [x3, x2, x1, x0] represent the coeffi cients b1, b2, and b3, respectively, of Eq. (18.7). Along with b0 5 f(x0) 5 0.0, Eq. (18.7) is

f3(x) 5 0 1 0.4620981(x 2 1) 2 0.05187311(x 2 1)(x 2 4)

1 0.007865529(x 2 1)(x 2 4)(x 2 6)

which can be used to evaluate f3(2) 5 0.6287686, which represents a relative error of �t 5 9.3%. The complete cubic polynomial is shown in Fig. 18.6.

f (x)

f (x) = ln x

f3(x)

True value

Cubic estimate

x50

2

0

1

FIGURE 18.6 The use of cubic interpolation to estimate ln 2.

18.1.4 Errors of Newton’s Interpolating Polynomials

Notice that the structure of Eq. (18.15) is similar to the Taylor series expansion in the sense that terms are added sequentially to capture the higher-order behavior of the underlying function. These terms are fi nite divided differences and, thus, represent

cha9792x_ch18_490-525.indd Page 497 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 497 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

498 INTERPOLATION

approximations of the higher-order derivatives. Consequently, as with the Taylor series, if the true underlying function is an nth-order polynomial, the nth-order interpolating polynomial based on n 1 1 data points will yield exact results. Also, as was the case with the Taylor series, a formulation for the truncation error can be obtained. Recall from Eq. (4.6) that the truncation error for the Taylor series could be expressed generally as

Rn 5 f (n11)(j)

(n 1 1)! (xi11 2 xi)

n11 (4.6)

where � is somewhere in the interval xi to xi11. For an nth-order interpolating polynomial, an analogous relationship for the error is

Rn 5 f (n11)(j)

(n 1 1)! (x 2 x0) (x 2 x1) p (x 2 xn) (18.16)

where � is somewhere in the interval containing the unknown and the data. For this formula to be of use, the function in question must be known and differentiable. This is not usually the case. Fortunately, an alternative formulation is available that does not require prior knowledge of the function. Rather, it uses a fi nite divided difference to approximate the (n 1 1)th derivative,

Rn 5 f [x, xn, xn21, p , x0] (x 2 x0) (x 2 x1) p (x 2 xn) (18.17)

where f [x, xn, xn21, . . . , x0] is the (n 1 1)th fi nite divided difference. Because Eq. (18.17) contains the unknown f(x), it cannot be solved for the error. However, if an additional data point f(xn11) is available, Eq. (18.17) can be used to estimate the error, as in

Rn > f [xn11, xn, xn21, p , x0] (x 2 x0) (x 2 x1) p (x 2 xn) (18.18)

EXAMPLE 18.4 Error Estimation for Newton’s Polynomial

Problem Statement. Use Eq. (18.18) to estimate the error for the second-order polyno- mial interpolation of Example 18.2. Use the additional data point f(x3) 5 f(5) 5 1.609438 to obtain your results.

Solution. Recall that in Example 18.2, the second-order interpolating polynomial provided an estimate of f2(2) 5 0.5658444, which represents an error of 0.6931472 2 0.5658444 5 0.1273028. If we had not known the true value, as is most usually the case, Eq. (18.18), along with the additional value at x3, could have been used to estimate the error, as in

R2 5 f [x3, x2, x1, x0] (x 2 x0) (x 2 x1) (x 2 x2)

or

R2 5 0.007865529(x 2 1)(x 2 4)(x 2 6)

where the value for the third-order fi nite divided difference is as computed previously in Example 18.3. This relationship can be evaluated at x 5 2 for

R2 5 0.007865529(2 2 1)(2 2 4)(2 2 6) 5 0.0629242

which is of the same order of magnitude as the true error.

cha9792x_ch18_490-525.indd Page 498 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 498 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 499

From the previous example and from Eq. (18.18), it should be clear that the error esti- mate for the nth-order polynomial is equivalent to the difference between the (n 1 1)th order and the nth-order prediction. That is,

Rn 5 fn11(x) 2 fn(x) (18.19)

In other words, the increment that is added to the nth-order case to create the (n 1 1)th- order case [that is, Eq. (18.18)] is interpreted as an estimate of the nth-order error. This can be clearly seen by rearranging Eq. (18.19) to give

fn11(x) 5 fn(x) 1 Rn

The validity of this approach is predicated on the fact that the series is strongly con- vergent. For such a situation, the (n 1 1)th-order prediction should be much closer to the true value than the nth-order prediction. Consequently, Eq. (18.19) conforms to our standard defi nition of error as representing the difference between the truth and an approximation. However, note that whereas all other error estimates for iterative approaches introduced up to this point have been determined as a present prediction minus a previous one, Eq. (18.19) represents a future prediction minus a present one. This means that for a series that is converging rapidly, the error estimate of Eq. (18.19) could be less than the true error. This would represent a highly unattractive quality if the error estimate were being employed as a stopping criterion. However, as will be described in the following section, higher-order interpolating polynomials are highly sensitive to data errors—that is, they are very ill-conditioned. When employed for in- terpolation, they often yield predictions that diverge signifi cantly from the true value. By “looking ahead” to sense errors, Eq. (18.19) is more sensitive to such divergence. As such, it is more valuable for the sort of exploratory data analysis for which Newton’s polynomial is best-suited.

18.1.5 Computer Algorithm for Newton’s Interpolating Polynomial

Three properties make Newton’s interpolating polynomials extremely attractive for com- puter applications:

1. As in Eq. (18.7), higher-order versions can be developed sequentially by adding a single term to the next lower-order equation. This facilitates the evaluation of several different-order versions in the same program. Such a capability is especially valuable when the order of the polynomial is not known a priori. By adding new terms se- quentially, we can determine when a point of diminishing returns is reached—that is, when addition of higher-order terms no longer signifi cantly improves the estimate or in certain situations actually detracts from it. The error equations discussed below in (3) are useful in devising an objective criterion for identifying this point of diminishing terms.

2. The fi nite divided differences that constitute the coeffi cients of the polynomial [Eqs. (18.8) through (18.11)] can be computed effi ciently. That is, as in Eq. (18.14) and Fig. 18.5, lower-order differences are used to compute higher-order differences. By utilizing this previously determined information, the coeffi cients can be computed effi ciently. The algorithm in Fig. 18.7 contains such a scheme.

3. The error estimate [Eq. (18.18)] can be very simply incorporated into a computer algorithm because of the sequential way in which the prediction is built.

cha9792x_ch18_490-525.indd Page 499 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 499 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

500 INTERPOLATION

All the above characteristics can be exploited and incorporated into a general algo- rithm for implementing Newton’s polynomial (Fig. 18.7). Note that the algorithm consists of two parts: The fi rst determines the coeffi cients from Eq. (18.7), and the second deter- mines the predictions and their associated error. The utility of this algorithm is demon- strated in the following example.

EXAMPLE 18.5 Error Estimates to Determine the Appropriate Order of Interpolation

Problem Statement. After incorporating the error [Eq. (18.18)], utilize the computer algorithm given in Fig. 18.7 and the following information to evaluate f(x) 5 ln x at x 5 2:

x f (x) � ln x

1 0 4 1.3862944 6 1.7917595 5 1.6094379 3 1.0986123 1.5 0.4054641 2.5 0.9162907 3.5 1.2527630

SUBROUTINE NewtInt (x, y, n, xi, yint, ea) LOCAL fddn,n DOFOR i 5 0, n fddi,0 5 yi END DO DOFOR j 5 1, n DOFOR i 5 0, n 2 j fddi,j 5 (fddi11,j21 2 fddi,j21)/(xi1j 2 xi) END DO END DO xterm 5 1 yint0 5 fdd0,0 DOFOR order 5 1, n xterm 5 xterm * (xi 2 xorder21) yint2 5 yintorder21 1 fdd0,order * xterm eaorder21 5 yint2 2 yintorder21 yintorder 5 yint2 END order END NewtInt

FIGURE 18.7 An algorithm for Newton’s interpolating polynomial written in pseudocode.

cha9792x_ch18_490-525.indd Page 500 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 500 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.1 NEWTON’S DIVIDED-DIFFERENCE INTERPOLATING POLYNOMIALS 501

Solution. The results of employing the algorithm in Fig. 18.7 to obtain a solution are shown in Fig. 18.8. The error estimates, along with the true error (based on the fact that ln 2 5 0.6931472), are depicted in Fig. 18.9. Note that the estimated error and the true error are similar and that their agreement improves as the order increases. From these results, it can be concluded that the fi fth-order version yields a good estimate and that higher-order terms do not signifi cantly enhance the prediction. This exercise also illustrates the importance of the positioning and ordering of the points. For example, up through the third-order estimate, the rate of improvement is slow because the points that are added (at x 5 4, 6, and 5) are distant and on one side of the point in question at x 5 2. The fourth-order estimate shows a somewhat greater improve- ment because the new point at x 5 3 is closer to the unknown. However, the most dramatic decrease in the error is associated with the inclusion of the fi fth-order term using the data point at x 5 1.5. Not only is this point close to the unknown but it is also positioned on the opposite side from most of the other points. As a consequence, the error is reduced by almost an order of magnitude. The signifi cance of the position and sequence of these data can also be illustrated by using the same data to obtain an estimate for ln 2 but considering the points in a different sequence. Figure 18.9 shows results for the case of reversing the order of the original data, that is, x0 5 3.5, x1 5 2.5, x3 5 1.5, and so forth. Because the initial points for this case are closer to and spaced on either side of ln 2, the error decreases much more rapidly than for the original situation. By the second-order term, the error has been reduced to less than �t 5 2%. Other combinations could be employed to obtain different rates of convergence.

NUMBER OF POINTS? 8 X( 0 ), y( 0 ) = ? 1,0 X( 1 ), y( 1 ) = ? 4,1.3862944 X( 2 ), y( 2 ) = ? 6,1.7917595 X( 3 ), y( 3 ) = ? 5,1.6094379 X( 4 ), y( 4 ) = ? 3,1.0986123 X( 5 ), y( 5 ) = ? 1.5,0.40546411 X( 6 ), y( 6 ) = ? 2.5,0.91629073 X( 7 ), y( 7 ) = ? 3.5,1.2527630

INTERPOLATION AT X = 2 ORDER F(X) ERROR 0 0.000000 0.462098 1 0.462098 0.103746 2 0.565844 0.062924 3 0.628769 0.046953 4 0.675722 0.021792 5 0.697514 -0.003616 6 0.693898 -0.000459 7 0.693439

FIGURE 18.8 The output of a program, based on the algorithm from Fig. 18.7 to evaluate ln 2.

cha9792x_ch18_490-525.indd Page 501 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 501 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

502 INTERPOLATION

The foregoing example illustrates the importance of the choice of base points. As should be intuitively obvious, the points should be centered around and as close as pos- sible to the unknown. This observation is also supported by direct examination of the error equation [Eq. (18.17)]. If we assume that the fi nite divided difference does not vary markedly along the range of these data, the error is proportional to the product: (x 2 x0) (x 2 x1) p (x 2 xn). Obviously, the closer the base points are to x, the smaller the magnitude of this product.

18.2 LAGRANGE INTERPOLATING POLYNOMIALS

The Lagrange interpolating polynomial is simply a reformulation of the Newton polyno- mial that avoids the computation of divided differences. It can be represented concisely as

fn(x) 5 a n

i50 Li(x) f(xi) (18.20)

FIGURE 18.9 Percent relative errors for the prediction of ln 2 as a function of the order of the interpolating polynomial.

Error

True error (original)

Estimated error (original)

Estimated error (reversed)

Order5

0.5

0

– 0.5

cha9792x_ch18_490-525.indd Page 502 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 502 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.2 LAGRANGE INTERPOLATING POLYNOMIALS 503

where

Li(x) 5 q n

j50 j?1

x 2 xj xi 2 xj

(18.21)

where P designates the “product of.” For example, the linear version (n 5 1) is

f1(x) 5 x 2 x1 x0 2 x1

f(x0) 1 x 2 x0 x1 2 x0

f(x1) (18.22)

and the second-order version is

f2(x) 5 (x 2 x1) (x 2 x2)

(x0 2 x1) (x0 2 x2) f(x0) 1

(x 2 x0) (x 2 x2)

(x1 2 x0) (x1 2 x2) f(x1)

1 (x 2 x0) (x 2 x1)

(x2 2 x0) (x2 2 x1) f(x2) (18.23)

Equation (18.20) can be derived directly from Newton’s polynomial (Box 18.1). However, the rationale underlying the Lagrange formulation can be grasped directly by realizing that each term Li(x) will be 1 at x 5 xi and 0 at all other sample points (Fig. 18.10). Thus, each product Li(x) f(xi) takes on the value of f(xi) at the sample point xi. Consequently, the summation of all the products designated by Eq. (18.20) is the unique nth-order polynomial that passes exactly through all n 1 1 data points.

EXAMPLE 18.6 Lagrange Interpolating Polynomials

Problem Statement. Use a Lagrange interpolating polynomial of the fi rst and second order to evaluate ln 2 on the basis of the data given in Example 18.2:

x0 5 1  f(x0) 5 0

x1 5 4  f(x1) 5 1.386294

x2 5 6  f(x2) 5 1.791760

Solution. The fi rst-order polynomial [Eq. (18.22)] can be used to obtain the estimate at x 5 2,

f1(2) 5 2 2 4

1 2 4 0 1

2 2 1

4 2 1 1.386294 5 0.4620981

In a similar fashion, the second-order polynomial is developed as [Eq. (18.23)]

f2(2) 5 (2 2 4)(2 2 6)

(1 2 4)(1 2 6) 0 1

(2 2 1)(2 2 6)

(4 2 1)(4 2 6) 1.386294

1 (2 2 1)(2 2 4)

(6 2 1)(6 2 4) 1.791760 5 0.5658444

As expected, both these results agree with those previously obtained using Newton’s interpolating polynomial.

cha9792x_ch18_490-525.indd Page 503 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 503 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

504 INTERPOLATION

Box 18.1 Derivation of the Lagrange Form Directly from Newton’s Interpolating Polynomial

The Lagrange interpolating polynomial can be derived directly from Newton’s formulation. We will do this for the fi rst-order case only [Eq. (18.2)]. To derive the Lagrange form, we reformulate the divided differences. For example, the fi rst divided difference,

f [x1, x0] 5 f (x1) 2 f (x0)

x1 2 x0 (B18.1.1)

can be reformulated as

f [x1, x0] 5 f (x1)

x1 2 x0 1

f (x0)

x0 2 x1 (B18.1.2)

which is referred to as the symmetric form. Substituting Eq. (B18.1.2) into Eq. (18.2) yields

f1(x) 5 f (x0) 1 x 2 x0 x1 2 x0

f (x1) 1 x 2 x0 x0 2 x1

f (x0)

Finally, grouping similar terms and simplifying yields the La- grange form,

f1(x) 5 x 2 x1 x0 2 x1

f (x0) 1 x 2 x0 x1 2 x0

f (x1)

FIGURE 18.10 A visual depiction of the rationale behind the Lagrange polynomial. This fi gure shows a second-order case. Each of the three terms in Eq. (18.23) passes through one of the data points and is zero at the other two. The summation of the three terms must, therefore, be the unique second-order polynomial f2(x) that passes exactly through the three points.

Summation of three terms = f2(x)

Third term

Second term

150

0

100

50

– 150

– 100

– 50

2015 3025

First term

cha9792x_ch18_490-525.indd Page 504 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 504 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.2 LAGRANGE INTERPOLATING POLYNOMIALS 505

Note that, as with Newton’s method, the Lagrange version has an estimated error of [Eq. (18.17)]

Rn 5 f [x, xn, xn21, p , x0]q n

i50 (x 2 xi)

Thus, if an additional point is available at x 5 xn11, an error estimate can be obtained. However, because the fi nite divided differences are not employed as part of the Lagrange algorithm, this is rarely done. Equations (18.20) and (18.21) can be very simply programmed for implementation on a computer. Figure 18.11 shows pseudocode that can be employed for this purpose. In summary, for cases where the order of the polynomial is unknown, the Newton method has advantages because of the insight it provides into the behavior of the different-order formulas. In addition, the error estimate represented by Eq. (18.18) can usually be integrated easily into the Newton computation because the estimate employs a fi nite difference (Example 18.5). Thus, for exploratory computations, Newton’s method is often preferable. When only one interpolation is to be performed, the Lagrange and Newton formula- tions require comparable computational effort. However, the Lagrange version is some- what easier to program. Because it does not require computation and storage of divided differences, the Lagrange form is often used when the order of the polynomial is known a priori.

EXAMPLE 18.7 Lagrange Interpolation Using the Computer

Problem Statement. We can use the algorithm from Fig. 18.11 to study a trend analysis problem associated with our now-familiar falling parachutist. Assume that we have

FUNCTION Lagrng(x, y, n, xx) sum 5 0 DOFOR i 5 0, n product 5 yi DOFOR j 5 0, n IF i � j THEN product 5 product*(xx 2 xj)/(xi 2 xj) ENDIF END DO sum 5 sum 1 product END DO Lagrng 5 sum END Lagrng

FIGURE 18.11 Pseudocode to implement Lagrange interpolation. This algorithm is set up to compute a single nth-order prediction, where n 1 1 is the number of data points.

cha9792x_ch18_490-525.indd Page 505 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 505 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

506 INTERPOLATION

developed instrumentation to measure the velocity of the parachutist. The measured data obtained for a particular test case are

Time, Measured Velocity v, s cm/s

1 800 3 2310 5 3090 7 3940 13 4755

Our problem is to estimate the velocity of the parachutist at t 5 10s to fi ll in the large gap in the measurements between t 5 7 and t 5 13s. We are aware that the behavior of interpolating polynomials can be unexpected. Therefore, we will construct polynomials of orders 4, 3, 2, and 1 and compare the results.

Solution. The Lagrange algorithm can be used to construct fourth-, third-, second-, and fi rst-order interpolating polynomials. The fourth-order polynomial and the input data can be plotted as shown in Fig. 18.12a. It is evident from this plot that the estimated value of y at x 5 10 is higher than the overall trend of these data. Figure 18.12b through d shows plots of the results of the computations for third-, second-, and fi rst-order interpolating polynomials, respectively. It is noted that the lower the order, the lower the estimated value of the velocity at t 5 10s. The plots of the in- terpolating polynomials indicate that the higher-order polynomials tend to overshoot the trend of these data. This suggests that the fi rst- or second-order versions are most ap- propriate for this particular trend analysis. It should be remembered, however, that be- cause we are dealing with uncertain data, regression would actually be more appropriate.

v, c

m /s

v, c

m /s

0 0

3000

6000

5 10 15

0 0

3000

6000

5 10 t(s)

15

0 0

3000

6000

5 10 15

0 0

3000

6000

5 10 t(s)

15

(a) (b)

(c) (d )

FIGURE 18.12 Plots showing (a) fourth-order, (b) third-order, (c) second-order, and (d) fi rst-order interpolations.

cha9792x_ch18_490-525.indd Page 506 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 506 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.4 INVERSE INTERPOLATION 507

The preceding example illustrates that higher-order polynomials tend to be ill- conditioned, that is, they tend to be highly sensitive to round-off error. The same problem applies to higher-order polynomial regression. Double-precision arithmetic sometimes helps mitigate the problem. However, as the order increases, there will come a point at which round-off error will interfere with the ability to interpolate using the simple approaches covered to this point.

18.3 COEFFICIENTS OF AN INTERPOLATING POLYNOMIAL

Although both the Newton and the Lagrange polynomials are well-suited for determining intermediate values between points, they do not provide a convenient polynomial of the conventional form

f(x) 5 a0 1 a1x 1 a2x 2 1 p 1 an  x n (18.24)

A straightforward method for computing the coeffi cients of this polynomial is based on the fact that n 1 1 data points are required to determine the n 1 1 coeffi cients. Thus, simultaneous linear algebraic equations can be used to calculate the a’s. For example, suppose that you desired to compute the coeffi cients of the parabola

f(x) 5 a0 1 a1x 1 a2 x 2 (18.25)

Three data points are required: [x0, f(x0)], [x1, f(x1)], and [x2, f(x2)]. Each can be substi- tuted into Eq. (18.25) to give

f(x0) 5 a0 1 a1x0 1 a2 x 2 0

f(x1) 5 a0 1 a1x1 1 a2 x 2 1 (18.26)

f(x2) 5 a0 1 a1x2 1 a2 x 2 2

Thus, for this case, the x’s are the knowns and the a’s are the unknowns. Because there are the same number of equations as unknowns, Eq. (18.26) could be solved by an elimination method from Part Three. It should be noted that the foregoing approach is not the most effi cient method that is available to determine the coeffi cients of an interpolating polynomial. Press et al. (2007) provide a discussion and computer codes for more effi cient approaches. Whatever technique is employed, a word of caution is in order. Systems such as Eq. (18.26) are notoriously ill-conditioned. Whether they are solved with an elimination method or with a more effi cient algorithm, the resulting coeffi cients can be highly inaccurate, particularly for large n. When used for a subsequent interpolation, they often yield erroneous results. In summary, if you are interested in determining an intermediate point, employ Newton or Lagrange interpolation. If you must determine an equation of the form of Eq. (18.24), limit yourself to lower-order polynomials and check your results carefully.

18.4 INVERSE INTERPOLATION

As the nomenclature implies, the f(x) and x values in most interpolation contexts are the dependent and independent variables, respectively. As a consequence, the values of the x’s are typically uniformly spaced. A simple example is a table of values derived for the

cha9792x_ch18_490-525.indd Page 507 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 507 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

508 INTERPOLATION

function f(x) 5 1yx,

x 1 2 3 4 5 6 7

f (x) 1 0.5 0.3333 0.25 0.2 0.1667 0.1429

Now suppose that you must use the same data, but you are given a value for f(x) and must determine the corresponding value of x. For instance, for the data above, sup- pose that you were asked to determine the value of x that corresponded to f(x) 5 0.3. For this case, because the function is available and easy to manipulate, the correct answer can be determined directly as x 5 1y0.3 5 3.3333. Such a problem is called inverse interpolation. For a more complicated case, you might be tempted to switch the f(x) and x values [that is, merely plot x versus f(x)] and use an approach like Lagrange interpolation to determine the result. Unfortunately, when you reverse the variables, there is no guarantee that the values along the new abscissa [the f(x)’s] will be evenly spaced. In fact, in many cases, the values will be “telescoped.” That is, they will have the appearance of a logarithmic scale with some adjacent points bunched together and others spread out widely. For example, for f(x) 5 1yx the result is

f (x) 0.1429 0.1667 0.2 0.25 0.3333 0.5 1

x 7 6 5 4 3 2 1

Such nonuniform spacing on the abscissa often leads to oscillations in the resulting interpolating polynomial. This can occur even for lower-order polynomials. An alternative strategy is to fi t an nth-order interpolating polynomial, fn(x), to the original data [that is, with f(x) versus x]. In most cases, because the x’s are evenly spaced, this polynomial will not be ill-conditioned. The answer to your problem then amounts to fi nding the value of x that makes this polynomial equal to the given f(x). Thus, the interpolation problem reduces to a roots problem! For example, for the problem outlined above, a simple approach would be to fi t a qua- dratic polynomial to the three points: (2, 0.5), (3, 0.3333) and (4, 0.25). The result would be

f2(x) 5 1.08333 2 0.375x 1 0.041667x 2

The answer to the inverse interpolation problem of fi nding the x corresponding to f(x) 5 0.3 would therefore involve determining the root of

0.3 5 1.08333 2 0.375x 1 0.041667x2

For this simple case, the quadratic formula can be used to calculate

x 5 0.375 6 2(20.375)2 2 4(0.041667)0.78333

2(0.041667) 5

5.704158

3.295842

Thus, the second root, 3.296, is a good approximation of the true value of 3.333. If additional accuracy were desired, a third- or fourth-order polynomial along with one of the root location methods from Part Two could be employed.

18.5 ADDITIONAL COMMENTS

Before proceeding to the next section, we must mention two additional topics: interpola- tion with equally spaced data and extrapolation.

cha9792x_ch18_490-525.indd Page 508 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 508 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.5 ADDITIONAL COMMENTS 509

Because both the Newton and Lagrange polynomials are compatible with arbitrarily spaced data, you might wonder why we address the special case of equally spaced data (Box 18.2). Prior to the advent of digital computers, these techniques had great utility for interpolation from tables with equally spaced arguments. In fact, a computational framework known as a divided-difference table was developed to facilitate the imple- mentation of these techniques. (Figure 18.5 is an example of such a table.) However, because the formulas are subsets of the computer-compatible Newton and Lagrange schemes and because many tabular functions are available as library subroutines, the need for the equispaced versions has waned. In spite of this, we have included them at this point because of their relevance to later parts of this book. In particular, they are needed to derive numerical integration formulas that typically employ equispaced data (Chap. 21). Because the numerical integration formulas have relevance to the solution of ordinary differential equations, the material in Box 18.2 also has signifi cance to Part Seven. Extrapolation is the process of estimating a value of f(x) that lies outside the range of the known base points, x0, x1, . . . , xn (Fig. 18.13). In a previous section, we mentioned that the most accurate interpolation is usually obtained when the unknown lies near the center of the base points. Obviously, this is violated when the unknown lies outside the range, and consequently, the error in extrapolation can be very large. As depicted in Fig. 18.13, the open-ended nature of extrapolation represents a step into the unknown because the process extends the curve beyond the known region. As such, the true curve could easily diverge from the prediction. Extreme care should, therefore, be exercised whenever a case arises where one must extrapolate.

FIGURE 18.13 Illustration of the possible divergence of an extrapolated prediction. The extrapolation is based on fi tting a parabola through the fi rst three known points.

f (x)

x

True curve

Extrapolation of interpolating polynomial

Interpolation Extrapolation

x2x1x0

cha9792x_ch18_490-525.indd Page 509 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 509 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

510 INTERPOLATION

Box 18.2 Interpolation with Equally Spaced Data

If data are equally spaced and in ascending order, then the indepen- dent variable assumes values of

x1 5 x0 1 h

x2 5 x0 1 2h

.

.

.

xn 5 x0 1 nh

where h is the interval, or step size, between these data. On this basis, the fi nite divided differences can be expressed in concise form. For example, the second forward divided difference is

f [x0, x1, x2] 5

f (x2) 2 f (x1)

x2 2 x1 2

f (x1) 2 f (x0)

x1 2 x0 x2 2 x0

which can be expressed as

f [x0, x1, x2] 5 f (x2) 2 2 f (x1) 1 f (x0)

2h2 (B18.2.1)

because x1 2 x0 5 x2 2 x1 5 (x2 2 x0)y2 5 h. Now recall that the second forward difference is equal to [numerator of Eq. (4.24)]

¢2 f (x0) 5 f (x2) 2 2 f (x1) 1 f (x0)

Therefore, Eq. (B18.2.1) can be represented as

f [x0, x1, x2] 5 ¢2 f (x0)

2!h2

or, in general,

f [x0, x1, p , xn] 5 ¢n f (x0)

n!hn (B18.2.2)

Using Eq. (B18.2.2), we can express Newton’s interpolating poly- nomial [Eq. (18.15)] for the case of equispaced data as

fn(x) 5 f (x0) 1 ¢ f (x0)

h (x 2 x0)

1 ¢2 f (x0)

2!h2 (x 2 x0) (x 2 x0 2 h)

1 p 1 ¢n f (x0)

n!hn (x 2 x0) (x 2 x0 2 h)

p [x 2 x0 2 (n 2 1)h] 1 Rn (B18.2.3)

where the remainder is the same as Eq. (18.16). This equation is known as Newton’s formula, or the Newton-Gregory forward for- mula. It can be simplifi ed further by defi ning a new quantity, �:

a 5 x 2 x0

h

This defi nition can be used to develop the following simplifi ed ex- pressions for the terms in Eq. (B18.2.3):

x 2 x0 5 ah

x 2 x0 2 h 5 ah 2 h 5 h(a 2 1)

.

.

.

x 2 x0 2 (n 2 1)h 5 ah 2 (n 2 1)h 5 h(a 2 n 1 1)

which can be substituted into Eq. (B18.2.3) to give

fn(x) 5 f (x0) 1 ¢ f (x0)a 1 ¢2 f (x0)

2! a(a 2 1)

1 p 1 ¢n f(x0)

n! a(a 2 1) p (a 2 n 1 1) 1 Rn

(B18.2.4)

where

Rn 5 f (n11)(j)

(n 1 1)! hn11a(a 2 1)(a 2 2) p (a 2 n)

This concise notation will have utility in our derivation and error analyses of the integration formulas in Chap. 21. In addition to the forward formula, backward and central Newton-Gregory formulas are also available. Carnahan, Luther, and Wilkes (1969) can be consulted for further information regard- ing interpolation for equally spaced data.

cha9792x_ch18_490-525.indd Page 510 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 510 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.6 SPLINE INTERPOLATION 511

18.6 SPLINE INTERPOLATION

In the previous sections, nth-order polynomials were used to interpolate between n 1 l data points. For example, for eight points, we can derive a perfect seventh-order poly- nomial. This curve would capture all the meanderings (at least up to and including seventh derivatives) suggested by the points. However, there are cases where these func- tions can lead to erroneous results because of round-off error and overshoot. An alterna- tive approach is to apply lower-order polynomials to subsets of data points. Such connecting polynomials are called spline functions. For example, third-order curves employed to connect each pair of data points are called cubic splines. These functions can be constructed so that the connections between adjacent cubic equations are visually smooth. On the surface, it would seem that the third-order approximation of the splines would be inferior to the seventh-order expres- sion. You might wonder why a spline would ever be preferable. Figure 18.14 illustrates a situation where a spline performs better than a higher- order polynomial. This is the case where a function is generally smooth but undergoes an abrupt change somewhere along the region of interest. The step increase depicted in Fig. 18.14 is an extreme example of such a change and serves to illustrate the point. Figure 18.14a through c illustrates how higher-order polynomials tend to swing through wild oscillations in the vicinity of an abrupt change. In contrast, the spline also connects the points, but because it is limited to lower-order changes, the oscillations are kept to a minimum. As such, the spline usually provides a superior approximation of the behavior of functions that have local, abrupt changes. The concept of the spline originated from the drafting technique of using a thin, fl exible strip (called a spline) to draw smooth curves through a set of points. The process is depicted in Fig. 18.15 for a series of fi ve pins (data points). In this technique, the drafter places paper over a wooden board and hammers nails or pins into the paper (and board) at the location of the data points. A smooth cubic curve results from interweaving the strip between the pins. Hence, the name “cubic spline” has been adopted for poly- nomials of this type. In this section, simple linear functions will fi rst be used to introduce some basic concepts and problems associated with spline interpolation. Then we derive an algorithm for fi tting quadratic splines to data. Finally, we present material on the cubic spline, which is the most common and useful version in engineering practice.

18.6.1 Linear Splines

The simplest connection between two points is a straight line. The fi rst-order splines for a group of ordered data points can be defi ned as a set of linear functions,

f(x) 5 f(x0) 1 m0(x 2 x0) x0 # x # x1 f(x) 5 f(x1) 1 m1(x 2 x1) x1 # x # x2    .

  .

  .

f(x) 5 f(xn21) 1 mn21(x 2 xn21) xn21 # x # xn

cha9792x_ch18_490-525.indd Page 511 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 511 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

512 INTERPOLATION

where mi is the slope of the straight line connecting the points:

mi 5 f(xi11) 2 f(xi)

xi11 2 xi (18.27)

These equations can be used to evaluate the function at any point between x0 and xn by fi rst locating the interval within which the point lies. Then the appropriate equation

(a)

f (x)

x0

(b)

f (x)

x0

(c)

f (x)

x0

(d)

f (x)

x0

FIGURE 18.14 A visual representation of a situation where the splines are superior to higher-order interpolating polynomials. The function to be fi t undergoes an abrupt increase at x 5 0. Parts (a) through (c) indicate that the abrupt change induces oscillations in interpolating polynomials. In contrast, because it is limited to third-order curves with smooth transitions, a linear spline (d) provides a much more acceptable approximation.

cha9792x_ch18_490-525.indd Page 512 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 512 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.6 SPLINE INTERPOLATION 513

is used to determine the function value within the interval. The method is obviously identical to linear interpolation.

EXAMPLE 18.8 First-Order Splines

Problem Statement. Fit the data in Table 18.1 with fi rst-order splines. Evaluate the function at x 5 5.

Solution. These data can be used to determine the slopes between points. For example, for the interval x 5 4.5 to x 5 7 the slope can be computed using Eq. (18.27):

m 5 2.5 2 7

7 2 4.5 5 0.60

The slopes for the other intervals can be computed, and the resulting fi rst-order splines are plotted in Fig. 18.16a. The value at x 5 5 is 1.3.

TABLE 18.1 Data to be fi t with spline functions.

x f(x)

3.0 2.5 4.5 1.0 7.0 2.5 9.0 0.5

FIGURE 18.15 The drafting technique of using a spline to draw smooth curves through a series of points. Notice how, at the end points, the spline straightens out. This is called a “natural” spline.

cha9792x_ch18_490-525.indd Page 513 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 513 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

514 INTERPOLATION

Visual inspection of Fig. 18.16a indicates that the primary disadvantage of fi rst- order splines is that they are not smooth. In essence, at the data points where two splines meet (called a knot), the slope changes abruptly. In formal terms, the fi rst derivative of the function is discontinuous at these points. This defi ciency is overcome by using higher- order polynomial splines that ensure smoothness at the knots by equating derivatives at these points, as discussed in the next section.

18.6.2 Quadratic Splines

To ensure that the mth derivatives are continuous at the knots, a spline of at least m 1 1 order must be used. Third-order polynomials or cubic splines that ensure continuous fi rst and second derivatives are most frequently used in practice. Although third and higher derivatives could be discontinuous when using cubic splines, they usually cannot be detected visually and consequently are ignored.

FIGURE 18.16 Spline fi ts of a set of four points. (a) Linear spline, (b) quadratic spline, and (c) cubic spline, with a cubic interpolating polynomial also plotted.

f (x)

x102 4 6

(a) 8

0

2

f (x)

x

(b)

0

2

f (x)

x

(c)

0

Interpolating cubic

First-order spline

Second-order spline

Cubic spline

2

cha9792x_ch18_490-525.indd Page 514 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 514 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.6 SPLINE INTERPOLATION 515

Because the derivation of cubic splines is somewhat involved, we have chosen to include them in a subsequent section. We have decided to fi rst illustrate the concept of spline interpolation using second-order polynomials. These “quadratic splines” have con- tinuous fi rst derivatives at the knots. Although quadratic splines do not ensure equal second derivatives at the knots, they serve nicely to demonstrate the general procedure for developing higher-order splines. The objective in quadratic splines is to derive a second-order polynomial for each in- terval between data points. The polynomial for each interval can be represented generally as

fi(x) 5 ai x 2 1 bi x 1 ci (18.28)

Figure 18.17 has been included to help clarify the notation. For n 1 1 data points (i 5 0, 1, 2, . . . , n), there are n intervals and, consequently, 3n unknown constants (the a’s, b’s, and c’s) to evaluate. Therefore, 3n equations or conditions are required to evaluate the unknowns. These are:

1. The function values of adjacent polynomials must be equal at the interior knots. This condition can be represented as

ai21 x 2 i21 1 bi21 xi21 1 ci21 5 f(xi21) (18.29)

ai x 2 i21 1 bi xi21 1 ci 5 f(xi21) (18.30)

for i 5 2 to n. Because only interior knots are used, Eqs. (18.29) and (18.30) each provide n 2 1 conditions for a total of 2n 2 2 conditions.

2. The fi rst and last functions must pass through the end points. This adds two additional equations:

a1x 2 0 1 b1x0 1 c1 5 f(x0) (18.31)

anx 2 n 1 bnxn 1 cn 5 f(xn) (18.32)

for a total of 2n 2 2 1 2 5 2n conditions.

FIGURE 18.17 Notation used to derive quadratic splines. Notice that there are n intervals and n 1 1 data points. The example shown is for n 5 3.

f (x)

f (x1)

f (x2)

f (x3)

f (x0)

xx0

a1x 2 + b1x + c1

a2x 2 + b2x + c2

a3x 2 + b3x + c3

x1 x2 x3 i = 0 i = 1 i = 2 i = 3

Interval 1 Interval 2 Interval 3

cha9792x_ch18_490-525.indd Page 515 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 515 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

516 INTERPOLATION

3. The fi rst derivatives at the interior knots must be equal. The fi rst derivative of Eq. (18.28) is

f ¿(x) 5 2ax 1 b

Therefore, the condition can be represented generally as

2ai21 xi21 1 bi21 5 2ai xi21 1 bi (18.33)

for i 5 2 to n. This provides another n 2 1 conditions for a total of 2n 1 n 2 1 5 3n 2 1. Because we have 3n unknowns, we are one condition short. Unless we have some additional information regarding the functions or their derivatives, we must make an arbitrary choice to successfully compute the constants. Although there are a number of different choices that can be made, we select the following:

4. Assume that the second derivative is zero at the fi rst point. Because the second derivative of Eq. (18.28) is 2ai, this condition can be expressed mathematically as

a1 5 0 (18.34)

The visual interpretation of this condition is that the fi rst two points will be con- nected by a straight line.

EXAMPLE 18.9 Quadratic Splines

Problem Statement. Fit quadratic splines to the same data used in Example 18.8 (Table 18.1). Use the results to estimate the value at x 5 5.

Solution. For the present problem, we have four data points and n 5 3 intervals. There- fore, 3(3) 5 9 unknowns must be determined. Equations (18.29) and (18.30) yield 2(3) 2 2 5 4 conditions:

20.25a1 1 4.5b1 1 c1 5 1.0

20.25a2 1 4.5b2 1 c2 5 1.0

49a2 1 7b2 1 c2 5 2.5

49a3 1 7b3 1 c3 5 2.5

Passing the fi rst and last functions through the initial and fi nal values adds 2 more [Eq. (18.31)]:

9a1 1 3b1 1 c1 5 2.5

and [Eq. (18.32)]

81a3 1 9b3 1 c3 5 0.5

Continuity of derivatives creates an additional 3 2 l 5 2 [Eq. (18.33)]:

9a1 1 b1 5 9a2 1 b2 14a2 1 b2 5 14a3 1 b3

Finally, Eq. (18.34) specifi es that a1 5 0. Because this equation specifi es a1 exactly, the problem reduces to solving eight simultaneous equations. These conditions can be

cha9792x_ch18_490-525.indd Page 516 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 516 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.6 SPLINE INTERPOLATION 517

expressed in matrix form as

H 4.5 1 0 0 0 0 0 0

0 0 20.25 4.5 1 0 0 0

0 0 49 7 1 0 0 0

0 0 0 0 0 49 7 1

3 1 0 0 0 0 0 0

0 0 0 0 0 81 9 1

1 0 29 21 0 0 0 0

0 0 14 1 0 214 21 0

X h b1 c1 a2 b2 c2 a3 b3 c3

x 5 h 1

1

2.5

2.5

2.5

0.5

0

0

x These equations can be solved using techniques from Part Three, with the results:

a1 5 0  b1 5 21  c1 5 5.5

a2 5 0.64  b2 5 26.76  c2 5 18.46

a3 5 21.6  b3 5 24.6  c3 5 291.3

which can be substituted into the original quadratic equations to develop the following relationships for each interval:

f1(x) 5 2x 1 5.5  3.0 # x # 4.5

f2(x) 5 0.64x 2 2 6.76x 1 18.46  4.5 # x # 7.0

f3(x) 5 21.6x 2 1 24.6x 2 91.3  7.0 # x # 9.0

When we use f2, the prediction for x 5 5 is, therefore,

f2(5) 5 0.64(5) 2 2 6.76(5) 1 18.46 5 0.66

The total spline fi t is depicted in Fig. 18.16b. Notice that there are two shortcomings that detract from the fi t: (1) the straight line connecting the fi rst two points and (2) the spline for the last interval seems to swing too high. The cubic splines in the next section do not exhibit these shortcomings and, as a consequence, are better methods for spline interpolation.

18.6.3 Cubic Splines

The objective in cubic splines is to derive a third-order polynomial for each interval between knots, as in

fi(x) 5 ai x 3 1 bi x

2 1 ci x 1 di (18.35)

Thus, for n 1 1 data points (i 5 0, 1, 2, . . . , n), there are n intervals and, consequently, 4n unknown constants to evaluate. Just as for quadratic splines, 4n conditions are required to evaluate the unknowns. These are:

1. The function values must be equal at the interior knots (2n 2 2 conditions). 2. The fi rst and last functions must pass through the end points (2 conditions).

cha9792x_ch18_490-525.indd Page 517 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 517 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

518 INTERPOLATION

3. The fi rst derivatives at the interior knots must be equal (n 2 1 conditions). 4. The second derivatives at the interior knots must be equal (n 2 1 conditions). 5. The second derivatives at the end knots are zero (2 conditions).

The visual interpretation of condition 5 is that the function becomes a straight line at the end knots. Specifi cation of such an end condition leads to what is termed a “natural” spline. It is given this name because the drafting spline naturally behaves in this fashion (Fig. 18.15). If the value of the second derivative at the end knots is nonzero (that is, there is some curvature), this information can be used alternatively to supply the two fi nal conditions. The above fi ve types of conditions provide the total of 4n equations required to solve for the 4n coeffi cients. Whereas it is certainly possible to develop cubic splines in this fashion, we will present an alternative technique that requires the solution of only n 2 1 equations. Although the derivation of this method (Box 18.3) is somewhat less straight- forward than that for quadratic splines, the gain in effi ciency is well worth the effort.

Box 18.3 Derivation of Cubic Splines

The fi rst step in the derivation (Cheney and Kincaid, 2008) is based on the observation that because each pair of knots is connected by a cubic, the second derivative within each interval is a straight line. Equation (18.35) can be differentiated twice to verify this observa- tion. On this basis, the second derivatives can be represented by a fi rst-order Lagrange interpolating polynomial [Eq. (18.22)]:

f i–(x) 5 f i–(xi21) x 2 xi

xi21 2 xi 1 f i–(xi)

x 2 xi21 xi 2 xi21

(B18.3.1)

where f i0(x) is the value of the second derivative at any point x within the ith interval. Thus, this equation is a straight line connect- ing the second derivative at the fi rst knot f 0(xi21) with the second derivative at the second knot f 0(xi). Next, Eq. (B18.3.1) can be integrated twice to yield an expres- sion for fi(x). However, this expression will contain two unknown constants of integration. These constants can be evaluated by in- voking the function-equality conditions—f(x) must equal f(xi21) at xi21 and f(x) must equal f(xi) at xi. By performing these evaluations, the following cubic equation results:

fi(x) 5 fi–(xi)

6(xi 2 xi21) (xi 2 x)

3 1 fi–(xi)

6(xi 2 xi21) (x 2 xi21)

3

1 c f (xi21) xi 2 xi21

2 f –(xi21) (xi 2 xi21)

6 d (xi 2 x)

1 c f (xi) xi 2 xi21

2 f –(xi) (xi 2 xi21)

6 d (x 2 xi21)

(B18.3.2)

Now, admittedly, this relationship is a much more complicated expression for the cubic spline for the ith interval than, say,

Eq. (18.35). However, notice that it contains only two unknown “coeffi cients,” the second derivatives at the beginning and the end of the interval—f 0(xi21) and f 0(xi). Thus, if we can deter- mine the proper second derivative at each knot, Eq. (B18.3.2) is a third- order polynomial that can be used to interpolate within the interval. The second derivatives can be evaluated by invoking the condi- tion that the fi rst derivatives at the knots must be continuous:

fi¿(xi) 5 fi¿11(xi) (B18.3.3)

Equation (B18.3.2) can be differentiated to give an expression for the fi rst derivative. If this is done for both the (i 2 1)th and the ith intervals and the two results are set equal according to Eq. (B18.3.3), the following relationship results:

(xi 2 xi21)f –(xi21) 1 2(xi11 2 xi21)f –(xi) 1 (xi11 2 xi)f –(xi11)

5 6

xi11 2 xi [ f (xi11) 2 f (xi) ]

1 6

xi 2 xi21 [ f (xi21) 2 f (xi) ] (B18.3.4)

If Eq. (B18.3.4) is written for all interior knots, n 2 1 simultaneous equations result with n 1 1 unknown second derivatives. However, because this is a natural cubic spline, the second derivatives at the end knots are zero and the problem reduces to n 2 1 equations with n 2 1 unknowns. In addition, notice that the system of equations will be tridiagonal. Thus, not only have we reduced the number of equations but we have also cast them in a form that is extremely easy to solve (recall Sec. 11.1.1).

cha9792x_ch18_490-525.indd Page 518 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 518 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.6 SPLINE INTERPOLATION 519

The derivation from Box 18.3 results in the following cubic equation for each interval:

fi(x) 5 f –i (xi21)

6(xi 2 xi21) (xi 2 x)

3 1 f –i (xi)

6(xi 2 xi21) (x 2 xi21)

3

1 c f(xi21) xi 2 xi21

2 f –(xi21) (xi 2 xi21)

6 d (xi 2 x)

1 c f(xi) xi 2 xi21

2 f –(xi)(xi 2 xi21)

6 d (x 2 xi21) (18.36)

This equation contains only two unknowns—the second derivatives at the end of each interval. These unknowns can be evaluated using the following equation:

(xi 2 xi21)f –(xi21) 1 2(xi11 2 xi21)f –(xi) 1 (xi11 2 xi)f –(xi11)

5 6

xi11 2 xi [

f(xi11) 2 f(xi)] 1

6 xi 2 xi11

[ f(xi11) 2 f(xi)] (18.37)

If this equation is written for all the interior knots, n 2 1 simultaneous equations result with n 2 1 unknowns. (Remember, the second derivatives at the end knots are zero.) The application of these equations is illustrated in the following example.

EXAMPLE 18.10 Cubic Splines

Problem Statement. Fit cubic splines to the same data used in Examples 18.8 and 18.9 (Table 18.1). Utilize the results to estimate the value at x 5 5.

Solution. The fi rst step is to employ Eq. (18.37) to generate the set of simultaneous equa- tions that will be utilized to determine the second derivatives at the knots. For example, for the fi rst interior knot, the following data are used:

x0 5 3   f(x0) 5 2.5

x1 5 4.5   f(x1) 5 1

x2 5 7   f(x2) 5 2.5

These values can be substituted into Eq. (18.37) to yield

(4.5 2 3)f –(3) 1 2(7 2 3)f –(4.5) 1 (7 2 4.5)f –(7)

5 6

7 2 4.5 (2.5 2 1) 1

6

4.5 2 3 (2.5 2 1)

Because of the natural spline condition, f 0(3) 5 0, and the equation reduces to

8f –(4.5) 1 2.5f –(7) 5 9.6

In a similar fashion, Eq. (18.37) can be applied to the second interior point to give

2.5f –(4.5) 1 9f –(7) 5 29.6

These two equations can be solved simultaneously for

f –(4.5) 5 1.67909 f –(7) 5 21.53308

cha9792x_ch18_490-525.indd Page 519 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 519 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

520 INTERPOLATION

These values can then be substituted into Eq. (18.36), along with values for the x’s and the f(x)’s, to yield

f1(x) 5 1.67909

6(4.5 2 3) (x 2 3)3 1

2.5

4.5 2 3 (4.5 2 x)

1 c 1 4.5 2 3

2 1.67909(4.5 2 3)

6 d (x 2 3)

or

f1(x) 5 0.186566(x 2 3) 3 1 1.666667(4.5 2 x) 1 0.246894(x 2 3)

This equation is the cubic spline for the fi rst interval. Similar substitutions can be made to develop the equations for the second and third intervals:

f2(x) 5 0.111939(7 2 x) 3 2 0.102205(x 2 4.5)3 2 0.299621(7 2 x)

1 1.638783(x 2 4.5)

and

f3(x) 5 20.127757(9 2 x) 3 1 1.761027(9 2 x) 1 0.25(x 2 7)

The three equations can then be employed to compute values within each interval. For example, the value at x 5 5, which falls within the second interval, is calculated as

f2(5) 5 0.111939(7 2 5) 3 2 0.102205(5 2 4.5)3 2 0.299621(7 2 5)

1 1.638783(5 2 4.5) 5 1.102886

Other values are computed and the results are plotted in Fig. 18.16c.

The results of Examples 18.8 through 18.10 are summarized in Fig. 18.16. Notice the progressive improvement of the fi t as we move from linear to quadratic to cubic splines. We have also superimposed a cubic interpolating polynomial on Fig. 18.16c. Although the cubic spline consists of a series of third-order curves, the resulting fi t dif- fers from that obtained using the third-order polynomial. This is due to the fact that the natural spline requires zero second derivatives at the end knots, whereas the cubic poly- nomial has no such constraint.

18.6.4 Computer Algorithm for Cubic Splines

The method for calculating cubic splines outlined in the previous section is ideal for computer implementation. Recall that, by some clever manipulations, the method reduced to solving n 2 1 simultaneous equations. An added benefi t of the derivation is that, as specifi ed by Eq. (18.37), the system of equations is tridiagonal. As described in Sec. 11.1, algorithms are available to solve such systems in an extremely effi cient manner. Figure 18.18 outlines a computational framework that incorporates these features. Note that the routine in Fig. 18.18 returns a single interpolated value, yu, for a given value of the dependent variable, xu. This is but one way in which spline interpolation

cha9792x_ch18_490-525.indd Page 520 28/10/13 8:08 PM f-479 cha9792x_ch18_490-525.indd Page 520 28/10/13 8:08 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.7 MULTIDIMENSIONAL INTERPOLATION 521

can be implemented. For example, you might want to determine the coeffi cients once, and then perform many interpolations. In addition, the routine returns both the fi rst (dy) and second (dy2) derivative at xu. Although it is not necessary to compute these quanti- ties, they prove useful in many applications of spline interpolation.

18.7 MULTIDIMENSIONAL INTERPOLATION

The interpolation methods for one-dimensional problems can be extended to multidimen- sional interpolation. In the present section, we will describe the simplest case of two- dimensional interpolation in Cartesian coordinates.

SUBROUTINE Spline (x,y,n,xu,yu,dy,d2y) LOCAL en, fn, gn, rn, d2xn CALL Tridiag(x,y,n,e,f,g,r) CALL Decomp(e,f,g,n21) CALL Subst(e,f,g,r,n21,d2x) CALL Interpol(x,y,n,d2x,xu,yu,dy,d2y) END Spline

SUBROUTINE Tridiag (x,y,n,e,f,g,r) f1 5 2 * (x22x0) g1 5 (x22x1) r1 5 6y(x22x1) * (y22y1) r1 5 r116y(x12x0) * (y02y1) DOFOR i 5 2, n22 ei 5 (xi2xi21) fi 5 2 * (xi11 2 xi21) gi 5 (xi11 2 xi) ri 5 6y(xi11 2 xi) * (yi11 2 yi) ri 5 ri16y(xi 2 xi21) * (yi21 2 yi) END DO en21 5 (xn21 2 xn22) fn21 5 2 * (xn 2 xn22) rn21 5 6y(xn 2 xn21) * (yn 2 yn21) rn21 5 rn21 1 6y(xn21 2 xn22) * (yn22 2 yn21) END Tridiag

SUBROUTINE Interpol (x,y,n,d2x,xu,yu,dy,d2y) flag 5 0 i 5 1 DO IF xu $ xi21 AND xu # xi THEN c1 5 d2xi21y6y(xi 2 xi21) c2 5 d2xiy6y(xi 2 xi21) c3 5 yi21y(xi 2 xi21) 2 d2xi21 * (xi2xi21)y6 c4 5 yiy(xi 2 xi21) 2 d2xi * (xi2xi21)y6 t1 5 c1 * (xi 2 xu)

3

t2 5 c2 * (xu 2 xi21) 3

t3 5 c3 * (xi 2 xu) t4 5 c4 * (xu 2 xi21) yu 5 t1 1 t2 1 t3 1 t4 t1 5 23 * c1 * (xi 2 xu)

2

t2 5 3 * c2 * (xu 2 xi21) 2

t3 5 2c3 t4 5 c4 dy 5 t1 1 t2 1 t3 1 t4 t1 5 6 * c1 * (xi 2 xu) t2 5 6 * c2 * (xu 2 xi21) d2y 5 t1 1 t2 flag 5 1 ELSE i 5 i 1 1 END IF IF i 5 n 1 1 OR flag 5 1 EXIT END DO IF flag 5 0 THEN PRINT “outside range” pause END IF END Interpol

FIGURE 18.18 Algorithm for cubic spline interpolation.

cha9792x_ch18_490-525.indd Page 521 28/10/13 8:09 PM f-479 cha9792x_ch18_490-525.indd Page 521 28/10/13 8:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

522 INTERPOLATION

18.7.1 Bilinear Interpolation

Two-dimensional interpolation deals with determining intermediate values for func- tions of two variables, z 5 f(xi, yi). As depicted in Fig. 18.19, we have values at four points: f(x1, y1), f(x2, y1), f(x1, y2), and f(x2, y2). We want to interpolate between these points to estimate the value at an intermediate point f(xi, yi). If we use a linear func- tion, the result is a plane connecting the points as in Fig. 18.19. Such functions are called bilinear. A simple approach for developing the bilinear function is depicted in Fig. 18.20. First, we can hold the y value fi xed and apply one-dimensional linear interpolation in the x direction. Using the Lagrange form, the result at (xi, y1) is

f(xi, y1) 5 xi 2 x2 x1 2 x2

f(x1, y1) 1 xi 2 x1 x2 2 x1

f(x2, y1) (18.38)

and at (xi, y2) is

f(xi, y2) 5 xi 2 x2 x1 2 x2

f(x1, y2) 1 xi 2 x1 x2 2 x1

f(x2, y2) (18.39)

These points can then be used to linearly interpolate along the y dimension to yield the fi nal result,

f(xi, yi) 5 yi 2 y2 y1 2 y2

f(xi, y1) 1 yi 2 y1 y2 2 y1

f(xi, y2) (18.40)

A single equation can be developed by substituting Eqs. (18.38) and (18.39) into Eq. (18.40) to give

FIGURE 18.19 Graphical depiction of two-dimensional bilinear interpolation where an intermediate value (fi lled circle) is estimated based on four given values (open circles).

x y

f (x1, y2)

f (xi, yi)

f(xi, yi)

f (x1, y1)

f (x2, y1)

f (x2, y2)

y2

y1 x1

x2

xi yi

f(x, y)

cha9792x_ch18_490-525.indd Page 522 28/10/13 8:09 PM f-479 cha9792x_ch18_490-525.indd Page 522 28/10/13 8:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

18.7 MULTIDIMENSIONAL INTERPOLATION 523

f(xi, yi) 5 xi 2 x2 x1 2 x2

yi 2 y2 y1 2 y2

f(x1, y1) 1 xi 2 x1 x2 2 x1

yi 2 y2 y1 2 y2

f(x2, y1)

1 xi 2 x2 x1 2 x2

yi 2 y1 y2 2 y1

f(x1, y2) 1 xi 2 x1 x2 2 x1

yi 2 y1 y2 2 y1

f(x2, y2) (18.41)

EXAMPLE 18.11 Bilinear Interpolation

Problem Statement. Suppose you have measured temperatures at a number of coordi- nates on the surface of a rectangular heated plate:

T(2, 1) 5 60  T(9, 1) 5 57.5

T(2, 6) 5 55  T(9, 6) 5 70

Use bilinear interpolation to estimate the temperature at xi 5 5.25 and yi 5 4.8.

Solution. Substituting these values into Eq. (18.41) gives

f(5.5, 4) 5 5.25 2 9

2 2 9 4.8 2 6

1 2 6 60 1

5.25 2 2

9 2 2 4.8 2 6

1 2 6 57.5

1 5.25 2 9

2 2 9 4.8 2 1

6 2 1 55 1

5.25 2 2

9 2 2 4.8 2 1

6 2 1 70 5 61.2143

Note that beyond the simple bilinear interpolation described in the foregoing example, higher-order polynomials and splines can also be used to interpolate in two dimensions. Further, these methods can be readily extended to three dimensions. We will return to this topic when we review software applications for interpolation at the end of Chap. 19.

FIGURE 18.20 Two-dimensional bilinear interpolation can be implemented by fi rst applying one-dimensional linear interpolation along the x dimension to determine values at xi. These values can then be used to linearly interpolate along the y dimension to yield the fi nal result at xi, yi.

f (xi, y1)

f (xi, y2)

f (xi , yi)

f (x1, y1)

f (x1, y2)

y1

x1 x2

yi

xi

y2

f (x2, y1)

f (x2, y2)

cha9792x_ch18_490-525.indd Page 523 28/10/13 8:09 PM f-479 cha9792x_ch18_490-525.indd Page 523 28/10/13 8:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

524 INTERPOLATION

PROBLEMS

18.1 Estimate the common logarithm of 10 using linear interpolation. (a) Interpolate between log 8 5 0.9030900 and log 12 5 1.0791812. (b) Interpolate between log 9 5 0.9542425 and log 11 5 1.0413927.

For each of the interpolations, compute the percent relative er- ror based on the true value.

18.2 Fit a second-order Newton’s interpolating polynomial to estimate log 10 using the data from Prob. 18.1 at x 5 8, 9, and 11. Compute the true percent relative error. 18.3 Fit a third-order Newton’s interpolating polynomial to estimate log 10 using the data from Prob. 18.1. 18.4 Repeat Probs. 18.1 through 18.3 using the Lagrange polynomial. 18.5 Given these data

x 1.6 2 2.5 3.2 4 4.5

f (x) 2 8 14 15 8 2

(a) Calculate f(2.8) using Newton’s interpolating polynomials of order 1 through 3. Choose the sequence of the points for your estimates to attain the best possible accuracy.

(b) Utilize Eq. (18.18) to estimate the error for each prediction. 18.6 Given these data

x 1 2 3 5 7 8

f (x) 3 6 19 99 291 444

Calculate f(4) using Newton’s interpolating polynomials of order 1 through 4. Choose your base points to attain good accuracy. What do your results indicate regarding the order of the polynomial used to generate the data in the table? 18.7 Repeat Prob. 18.6 using Lagrange polynomials of order 1 through 3. 18.8 The following data come from a table that was measured with high precision. Use the best numerical method (for this type of prob- lem) to determine y at x 5 3.5. Note that a polynomial will yield an exact value. Your solution should prove that your result is exact.

x 0 1.8 5 6 8.2 9.2 12

y 26 16.415 5.375 3.5 2.015 2.54 8

18.9 Use Newton’s interpolating polynomial to determine y at x 5 3.5 to the best possible accuracy. Compute the fi nite divided differ- ences as in Fig. 18.5 and order your points to attain optimal accu- racy and convergence.

x 0 1 2.5 3 4.5 5 6

y 2 5.4375 7.3516 7.5625 8.4453 9.1875 12

18.10 Use Newton’s interpolating polynomial to determine y at x  5 8 to the best possible accuracy. Compute the fi nite divided

differences as in Fig. 18.5 and order your points to attain optimal accuracy and convergence.

x 0 1 2 5.5 11 13 16 18

y 0.5 3.134 5.3 9.9 10.2 9.35 7.2 6.2

18.11 Employ inverse interpolation using a cubic interpolating polynomial and bisection to determine the value of x that corre- sponds to f(x) 5 0.23 for the following tabulated data:

x 2 3 4 5 6 7

y 0.5 0.3333 0.25 0.2 0.1667 0.1429

18.12 Employ inverse interpolation to determine the value of x that corresponds to f(x) 5 0.85 for the following tabulated data:

x 0 1 2 3 4 5

f (x) 0 0.5 0.8 0.9 0.941176 0.961538

Note that the values in the table were generated with the function f(x) 5 x2y(1 1 x2). (a) Determine the correct value analytically. (b) Use cubic interpolation of x versus y. (c) Use inverse interpolation with quadratic interpolation and the

quadratic formula. (d) Use inverse interpolation with cubic interpolation and bisec-

tion. For parts (b) through (d) compute the true percent relative error.

18.13 Develop quadratic splines for the fi rst fi ve data points in Prob. 18.5 and predict f(3.4) and f(2.2). 18.14 Develop cubic splines for the data in Prob. 18.6 and (a) pre- dict f(4) and f(2.5) and (b) verify that f2(3) and f3(3) 5 19. 18.15 Determine the coeffi cients of the parabola that passes through the last three points in Prob. 18.5. 18.16 Determine the coeffi cients of the cubic equation that passes through the fi rst four points in Prob. 18.6. 18.17 Develop, debug, and test a program in either a high-level language or macro language of your choice to implement Newton’s interpolating polynomial based on Fig. 18.7. 18.18 Test the program you developed in Prob. 18.17 by duplicat- ing the computation from Example 18.5. 18.19 Use the program you developed in Prob. 18.17 to solve Probs. 18.1 through 18.3. 18.20 Use the program you developed in Prob. 18.17 to solve Probs. 18.5 and 18.6. Utilize all the data to develop fi rst- through fi fth-order polynomials. For both problems, plot the estimated error versus order. 18.21 Develop, debug, and test a program in either a high-level language or macro language of your choice to implement Lagrange

cha9792x_ch18_490-525.indd Page 524 28/10/13 8:09 PM f-479 cha9792x_ch18_490-525.indd Page 524 28/10/13 8:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 525

(c) Use the fi ve points from (b) to estimate f(0.8) with fi rst- through fourth-order Newton interpolating polynomials.

(d) Generate and plot a cubic spline using the fi ve points from (b). (e) Discuss your results. 18.27 The following is the built-in humps function that MATLAB uses to demonstrate some of its numerical capabilities:

f (x) 5 1

(x 2 0.3)2 1 0.01 1

1

(x 2 0.9)2 1 0.04 2 6

The humps function exhibits both fl at and steep regions over a relatively short x range. Generate values of this function at inter- vals of 0.1 over the range from x 5 0 to 1. Fit these data with a cubic spline and create a plot comparing the fi t with the exact humps function. 18.28 The following data defi ne the sea-level concentration of dissolved oxygen for fresh water as a function of temperature:

T,°C 0 8 16 24 32 40

o,mg/L 14.621 11.843 9.870 8.418 7.305 6.413

Estimate o(27) using (a) linear interpolation, (b) Newton’s interpo- lating polynomial, and (c) cubic splines. Note that the exact result is 7.986 mg/L. 18.29 Generate eight equally-spaced points from the function

f(t) 5 sin2t

from t 5 0 to 2p. Fit these data with (a) a seventh-order interpolat- ing polynomial and (b) a cubic spline. 18.30 Temperatures are measured at various points on a heated plate (Table P18.30). Estimate the temperature at (a) x 5 4, y 5 3.2, and (b) x 5 4.3, y 5 2.7.

TABLE P18.30 Temperature (8C) at various points on a square heated plate.

x � 0 x � 2 x � 4 x � 6 x � 8

y 5 0 100.00 90.00 80.00 70.00 60.00 y 5 2 85.00 64.49 53.50 48.15 50.00 y 5 4 70.00 48.90 38.43 35.03 40.00 y 5 6 55.00 38.78 30.39 27.07 30.00 y 5 8 40.00 35.00 30.00 25.00 20.00

interpolation. Base it on the pseudocode from Fig. 18.11. Test it by duplicating Example 18.7. 18.22 A useful application of Lagrange interpolation is called a table look-up. As the name implies, this involves “looking-up” an intermediate value from a table. To develop such an algo- rithm, the table of x and f(x) values are fi rst stored in a pair of one-dimensional arrays. These values are then passed to a func- tion along with the x value you wish to evaluate. The function then performs two tasks. First, it loops down through the table until it fi nds the interval within which the unknown lies. Then it applies a technique like Lagrange interpolation to determine the proper f(x) value. Develop such a function using a cubic Lagrange polynomial to perform the interpolation. For intermedi- ate intervals, this is a nice choice because the unknown will be located in the interval in the middle of the four points necessary to generate the cubic. For the fi rst and last intervals, use a qua- dratic Lagrange polynomial. Also have your code detect when the user requests a value outside the range of x’s. For such cases, the function should display an error message. Test your program for f(x) 5 ln x using data from x 5 1, 2, p , 10. 18.23 Develop, debug, and test a program in either a high-level language or macro language of your choice to implement cubic spline interpolation based on Fig. 18.18. Test the program by dupli- cating Example 18.10. 18.24 Use the software developed in Prob. 18.23 to fi t cubic splines through the data in Probs. 18.5 and 18.6. For both cases, predict f(2.25). 18.25 Use the portion of the given steam table for superheated H2O at 200 MPa to (a) fi nd the corresponding entropy s for a spe- cifi c volume v of 0.108 m3/kg with linear interpolation, (b) fi nd the same corresponding entropy using quadratic interpolation, and (c) fi nd the volume corresponding to an entropy of 6.6 using in- verse interpolation.

v(m3/kg) 0.10377 0.11144 0.1254

s(kJ/kg ? K) 6.4147 6.5453 6.7664

18.26 Runge’s function is written as

f(x) 5 1

1 1 25x2

(a) Develop a plot of this function for the interval from x 5 21 to 1. (b) Generate and plot the fourth-order Lagrange interpolating

polynomial using equispaced function values corresponding to x 5 21, 20.5, 0, 0.5, and 1.

cha9792x_ch18_490-525.indd Page 525 28/10/13 8:09 PM f-479 cha9792x_ch18_490-525.indd Page 525 28/10/13 8:09 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19 C H A P T E R 19

526

Fourier Approximation

To this point, our presentation of interpolation has emphasized standard polynomials—that is, linear combinations of the monomials 1, x, x2, . . . , xm (Fig. 19.1a). We now turn to another class of functions that has immense importance in engineering. These are the trigonometric functions 1, cos x, cos 2x, . . . , cos nx, sin x, sin 2x, . . . , sin nx (Fig. 19.1b). Engineers often deal with systems that oscillate or vibrate. As might be expected, trigonometric functions play a fundamental role in modeling such problem contexts.

FIGURE 19.1 The fi rst fi ve (a) monomials and (b) trigonometric functions. Note that for the intervals shown, both types of function range in value between 21 and 1. However, notice that the peak values for the monomials all occur at the extremes whereas for the trigonometric functions the peaks are more uniformly distributed across the interval.

x4 x3

x2 xx2

x4

x3

f(x)

x1– 1

1

cos t

sin 2t

cos 2t

sin t

cos t

sin t

cos 2t

sin 2t

t

f(x)

�– �

1

(a)

(b)

cha9792x_ch19_526-562.indd Page 526 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 526 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.1 CURVE FITTING WITH SINUSOIDAL FUNCTIONS 527

Fourier approximation represents a systematic framework for using trigonometric series for this purpose. One of the hallmarks of a Fourier analysis is that it deals with both the time and the frequency domains. Because some engineers are not comfortable with the latter, we have devoted a large fraction of the subsequent material to a general overview of Fourier approximation. An important aspect of this overview will be to familiarize you with the frequency domain. This orientation is then followed by an introduction to numerical methods for computing discrete Fourier transforms.

19.1 CURVE FITTING WITH SINUSOIDAL FUNCTIONS

A periodic function f(t) is one for which

f(t) 5 f(t 1 T) (19.1)

FIGURE 19.2 Aside from trigonometric functions such as sines and cosines, periodic functions include waveforms such as (a) the square wave and (b) the sawtooth wave. Beyond these idealized forms, periodic signals in nature can be (c) non- ideal and (d) contaminated by noise. The trigonometric func- tions can be used to represent and to analyze all these cases.

T

(a)

T

(b)

T

(d)

T

(c)

cha9792x_ch19_526-562.indd Page 527 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 527 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

528 FOURIER APPROXIMATION

where T is a constant called the period that is the smallest value for which Eq. (19.1) holds. Common examples include waveforms such as square and sawtooth waves (Fig. 19.2). The most fundamental are sinusoidal functions. In the present discussion, we will use the term sinusoid to represent any waveform that can be described as a sine or cosine. There is no clear-cut convention for choosing either function, and in any case, the results will be identical. For this chapter, we will use the cosine, which is expressed generally as

f(t) 5 A0 1 C1 cos(v0t 1 u) (19.2)

Thus, four parameters serve to characterize the sinusoid (Fig. 19.3). The mean value A0 sets the average height above the abscissa. The amplitude C1 specifi es the height of the

FIGURE 19.3 (a) A plot of the sinusoidal function y(t) 5 A0 1 C1 cos(v0t 1 u). For this case, A0 5 1.7, C1 5 1, v0 5 2p/T 5 2p/(1.5 s), and u 5 p/3 radians 5 1.0472 (5 0.25 s). Other parameters used to describe the curve are the frequency f 5 v0/(2p), which for this case is 1 cycle/(1.5 s) and the period T 5 1.5 s. (b) An alternative expression of the same curve is y(t ) 5 A0 1 A1 cos(v0t ) 1 B1 sin(v0t ). The three components of this function are depicted in (b), where A1 5 0.5 and B1 5 20.866. The summation of the three curves in (b) yields the single curve in (a).

t, s

C1

A0 T

y(t)

�t, rad3�

2

1

2

1

2�

�0

1

0

– 1

2 A0

B1 sin (�0t)

A1 cos (�0t)

(a)

(b)

cha9792x_ch19_526-562.indd Page 528 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 528 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.1 CURVE FITTING WITH SINUSOIDAL FUNCTIONS 529

oscillation. The angular frequency v0 characterizes how often the cycles occur. Finally, the phase angle, or phase shift, u parameterizes the extent to which the sinusoid is shifted horizontally. It can be measured as the distance in radians from t 5 0 to the point at which the cosine function begins a new cycle. As depicted in Fig. 19.4a, a negative value is referred to as a lagging phase angle because the curve cos(v0t 2 u) begins a new cycle u radians after cos(v0t). Thus, cos(v0t 2 u) is said to lag cos(v0t). Conversely, as in Fig. 19.4b, a positive value is referred to as a leading phase angle. Note that the angular frequency (in radians/time) is related to frequency f (in cycles/ time) by

v0 5 2pf (19.3)

and frequency in turn is related to period T (in units of time) by

f 5 1

T (19.4)

Although Eq. (19.2) is an adequate mathematical characterization of a sinusoid, it is awkward to work with from the standpoint of curve fi tting because the phase shift is included in the argument of the cosine function. This defi ciency can be overcome by invoking the trigonometric identity

C1 cos(v0t 1 u) 5 C1[cos(v0t) cos(u) 2 sin(v0t) sin(u)] (19.5)

FIGURE 19.4 Graphical depictions of (a) a lagging phase angle and (b) a leading phase angle. Note that the lagging curve in (a) can be alternatively described as cos(v0t 1 3p/2). In other words, if a curve lags by an angle of a, it can also be represented as leading by 2p 2 a.

cos (�0t)

t

cos �0t – �

2

cos �0t + �

2 cos (�0t)

t

(a)

(b)

cha9792x_ch19_526-562.indd Page 529 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 529 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

530 FOURIER APPROXIMATION

Substituting Eq. (19.5) into Eq. (19.2) and collecting terms gives (Fig. 19.3b)

f(t) 5 A0 1 A1 cos(v0t) 1 B1 sin(v0t) (19.6)

where

A1 5 C1 cos(u)  B1 5 2C1 sin(u) (19.7)

Dividing the two parts of Eq. (19.7) gives

u 5 arctan a2B1 A1 b (19.8)

where, if A1 , 0, add p to u. Squaring and summing Eq. (19.7) leads to

C1 5 2A 2 1 1 B

2 1 (19.9)

Thus, Eq. (19.6) represents an alternative formulation of Eq. (19.2) that still requires four parameters but that is cast in the format of a general linear model [recall Eq. (17.23)]. As we will discuss in the next section, it can be simply applied as the basis for a least- squares fi t. Before proceeding to the next section, however, we should stress that we could have employed a sine rather than a cosine as our fundamental model of Eq. (19.2). For example,

f(t) 5 A0 1 C1 sin(v0t 1 d)

could have been used. Simple relationships can be applied to convert between the two forms

sin(v0t 1 d) 5 cos av0t 1 d 2 p2 b and

cos(v0t 1 u) 5 sin av0t 1 u 1 p2 b (19.10) In other words, u 5 d 2 p/2. The only important consideration is that one or the other format should be used consistently. Thus, we will use the cosine version through- out our discussion.

19.1.1 Least-Squares Fit of a Sinusoid

Equation (19.6) can be thought of as a linear least-squares model

y 5 A0 1 A1 cos(v0t) 1 B1 sin(v0t) 1 e (19.11)

which is just another example of the general model [recall Eq. (17.23)]

y 5 a0z0 1 a1z1 1 a2z2 1 p 1 am zm 1 e (17.23)

where z0 5 1, zl 5 cos(v0t), z2 5 sin(v0t), and all other z’s 5 0. Thus, our goal is to determine coeffi cient values that minimize

Sr 5 a N

i51 {yi 2 [A0 1 A1 cos(v0ti) 1 B1 sin(v0ti)]}

2

cha9792x_ch19_526-562.indd Page 530 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 530 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.1 CURVE FITTING WITH SINUSOIDAL FUNCTIONS 531

The normal equations to accomplish this minimization can be expressed in matrix form as [recall Eq. (17.25)]

£ N gcos(v0t) gsin(v0t)

gcos(v0t) gcos 2(v0t) gcos(v0t) sin(v0t)

gsin(v0t) gcos(v0t) sin(v0t) gsin 2(v0t)

§ • A0 A1 B0

5 • gy

gy cos(v0t) gy sin(v0t)

¶ (19.12)

These equations can be employed to solve for the unknown coeffi cients. However, rather than do this, we can examine the special case where there are N observations equispaced at intervals of Dt and with a total record length of T 5 (N 2 1) Dt. For this situation, the following average values can be determined (see Prob. 19.1):

gsin(v0t) N

5 0    gcos(v0t)

N 5 0

gsin2(v0t) N

5 1

2   

gcos2(v0t) N

5 1

2 (19.13)

gcos(v0t) sin(v0t)

N 5 0

Thus, for equispaced points the normal equations become

£ N 0 0

0 Ny2 0 0 0 Ny2

§ • A0 A1 B1

¶ 5 • gy

gy cos(v0t) gy sin(v0t)

The inverse of a diagonal matrix is merely another diagonal matrix whose elements are the reciprocals of the original. Thus, the coeffi cients can be determined as

• A0 A1 B1

¶ 5 £ 1yN 0 0

0 2yN 0 0 0 2yN

§ • gy

gy cos(v0t) gy sin(v0t)

or

A0 5 gy N

(19.14)

A1 5 2

N gy cos(v0t) (19.15)

B1 5 2

N gy sin(v0t) (19.16)

EXAMPLE 19.1 Least-Squares Fit of a Sinusoid

Problem Statement. The curve in Fig. 19.3 is described by y 5 1.7 1 cos(4.189t 1 1.0472). Generate 10 discrete values for this curve at intervals of Dt 5 0.15 for the range

cha9792x_ch19_526-562.indd Page 531 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 531 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

532 FOURIER APPROXIMATION

t 5 0 to 1.35. Use this information to evaluate the coeffi cients of Eq. (19.11) by a least- squares fi t.

Solution. The data required to evaluate the coeffi cients with v 5 4.189 are

t y y cos(V0t) y sin(V0t)

0 2.200 2.200 0.000 0.15 1.595 1.291 0.938 0.30 1.031 0.319 0.980 0.45 0.722 20.223 0.687 0.60 0.786 20.636 0.462 0.75 1.200 21.200 0.000 0.90 1.805 21.460 21.061 1.05 2.369 20.732 22.253 1.20 2.678 0.829 22.547 1.35 2.614 2.114 21.536

S5 17.000 2.502 24.330

These results can be used to determine [Eqs. (19.14) through (19.16)]

A0 5 17.000

10 5 1.7  A1 5

2

10 2.502 5 0.500  B1 5

2

10 (24.330) 5 20.866

Thus, the least-squares fi t is

y 5 1.7 1 0.500 cos(v0t) 2 0.866 sin(v0t)

The model can also be expressed in the format of Eq. (19.2) by calculating [Eq. (19.8)]

u 5 arctan a220.866 0.500

b 5 1.0472 and [Eq. (19.9)]

C1 5 2(0.5) 2 1 (20.866)2 5 1.00

to give

y 5 1.7 1 cos(v0t 1 1.0472)

or alternatively, as a sine by using [Eq. (19.10)]

y 5 1.7 1 sin(v0t 1 2.618)

The foregoing analysis can be extended to the general model

f(t) 5 A0 1 A1 cos(v0t) 1 B1 sin(v0t) 1 A2 cos(2v0t) 1 B2 sin(2v0t)

1 p 1 Am cos(mv0t) 1 Bm sin(mv0t)

cha9792x_ch19_526-562.indd Page 532 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 532 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.2 CONTINUOUS FOURIER SERIES 533

where, for equally spaced data, the coeffi cients can be evaluated by

A0 5 gy N

Aj 5 2

N gy cos( jv0t)

Bj 5 2

N gy sin( jv0t)

s  j 5 1, 2, p , m Although these relationships can be used to fi t data in the regression sense (that is, N . 2m 1 1), an alternative application is to employ them for interpolation or colloca- tion—that is, to use them for the case where the number of unknowns, 2m 1 1, is equal to the number of data points, N. This is the approach used in the continuous Fourier series, as described next.

19.2 CONTINUOUS FOURIER SERIES

In the course of studying heat-fl ow problems, Fourier showed that an arbitrary periodic function can be represented by an infi nite series of sinusoids of harmonically related frequencies. For a function with period T, a continuous Fourier series can be written1

f(t) 5 a0 1 a1 cos(v0t) 1 b1 sin(v0t) 1 a2 cos(2v0t) 1 b2 sin(2v0t) 1 p

or more concisely,

f(t) 5 a0 1 a q

k51 [ak cos(kv0t) 1 bk sin(kv0t)] (19.17)

where v0 5 2pyT is called the fundamental frequency and its constant multiples 2v0, 3v0, etc., are called harmonics. Thus, Eq. (19.17) expresses f(t) as a linear combination of the basis functions: 1, cos(v0t), sin(v0t), cos(2v0t), sin(2v0t), . . . . As described in Box 19.1, the coeffi cients of Eq. (19.17) can be computed via

ak 5 2

T #

T

0

f(t) cos(kv0t) dt (19.18)

and

bk 5 2

T #

T

0 f(t) sin(kv0t) dt (19.19)

for k 5 1, 2, . . . and

a0 5 1

T #

T

0 f(t) dt (19.20)

1The existence of the Fourier series is predicated on the Dirichlet conditions. These specify that the periodic function have a fi nite number of maxima and minima and that there be a fi nite number of jump discontinuities. In general, all physically derived periodic functions satisfy these conditions.

cha9792x_ch19_526-562.indd Page 533 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 533 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

534 FOURIER APPROXIMATION

Box 19.1 Determination of the Coeffi cients of the Continuous Fourier Series

As was done for the discrete data of Sec. 19.1.1, the following rela- tionships can be established:

# T

0 sin(kv0t) dt 5 #

T

0 cos(kv0t) dt 5 0 (B19.1.1)

# T

0 cos(kv0t) sin(gv0t) dt 5 0 (B19.1.2)

# T

0 sin(kv0t) sin(gv0t) dt 5 0 (B19.1.3)

# T

0 cos(kv0t) cos(gv0t) dt 5 0 (B19.1.4)

# T

0 sin2(kv0t) dt 5 #

T

0 cos2(kv0t) dt 5

T

2 (B19.1.5)

To evaluate its coeffi cients, each side of Eq. (19.17) can be inte- grated to give

# T

0 f(t) dt 5 #

T

0 a0 dt 1 #

T

0 a q

k51 [ak cos(kv0t)

1 bk sin(kv0t) ] dt

Because every term in the summation is of the form of Eq. (B19.1.1), the equation becomes

# T

0 f(t) dt 5 a0T

which can be solved for

a0 5 eT0 f (t) dt

T

Thus, a0 is simply the average value of the function over the period. To evaluate one of the cosine coeffi cients, for example, am, Eq. (19.17) can be multiplied by (mv0t) and integrated to give

# T

0 f(t) cos(mv0t) dt 5 #

T

0 a0 cos(mv0t) dt

1 # T

0 a q

k51 ak cos(kv0t) cos(mv0t) dt

1 # T

0 a q

k51 bk sin(kv0t) cos(mv0t) dt (B19.1.6)

From Eqs. (B19.1.1), (B19.1.2), and (B19.1.4), we see that every term on the right-hand side is zero, with the exception of the case where k 5 m. This latter case can be evaluated by Eq. (B19.1.5) and, therefore, Eq. (B19.1.6) can be solved for am, or more gener- ally [Eq. (19.18)],

ak 5 2

T #

T

0 f (t) cos(kv0t) dt

for k 5 1, 2, . . . . In a similar fashion, Eq. (19.17) can be multiplied by sin(mv0t), integrated, and manipulated to yield Eq. (19.19).

EXAMPLE 19.2 Continuous Fourier Series Approximation

Problem Statement. Use the continuous Fourier series to approximate the square or rectangular wave function (Fig. 19.5)

f(t) 5 • 21 2Ty2 , t , 2Ty4 1 2Ty4 , t , Ty4 21 Ty4 , t , Ty2

Solution. Because the average height of the wave is zero, a value of a0 5 0 can be obtained directly. The remaining coeffi cients can be evaluated as [Eq. (19.18)]

ak 5 2

T #

Ty2

2Ty2 f(t) cos(kv 0 t) dt

5 2

T c2#

2Ty4

2Ty2

cos(kv0t) dt 1 # Ty4

2Ty4

cos(kv0t) dt 2 # Ty2

Ty4

cos(kv0t) dt d

cha9792x_ch19_526-562.indd Page 534 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 534 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.2 CONTINUOUS FOURIER SERIES 535

FIGURE 19.5 A square or rectangular wave- form with a height of 2 and a period T 5 2p/v0.

1

– T/2 T/20– T T

–1

(a)

(b)

(c)

cos (�0t) 4 �

cos (3�0t) 4

3�

cos (5�0t) 4

5�

FIGURE 19.6 The Fourier series approxima- tion of the square wave from Fig. 19.5. The series of plots shows the summation up to and including the (a) fi rst, (b) second, and (c) third terms. The individual terms that were added at each stage are also shown.

cha9792x_ch19_526-562.indd Page 535 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 535 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

536 FOURIER APPROXIMATION

The integrals can be evaluated to give

ak 5 • 4y(kp) for k 5 1, 5, 9, p 24y(kp) for k 5 3, 7, 11, p 0 for k 5 even integers

Similarly, it can be determined that all the b’s 5 0. Therefore, the Fourier series approximation is

f(t) 5 4 p

cos(v0t) 2 4

3p cos(3v0t) 1

4

5p cos(5v0t) 2

4

7p cos(7v0t) 1 p

The results up to the fi rst three terms are shown in Fig. 19.6. It should be mentioned that the square wave in Fig. 19.5 is called an even function because f(t) 5 f(2t). Another example of an even function is cos(t). It can be shown (Van Valkenburg, 1974) that the b’s in the Fourier series always equal zero for even functions. Note also that odd functions are those for which f(t) 5 2f(2t). The function sin(t) is an odd function. For this case, the a’s will equal zero.

Aside from the trigonometric format of Eq. (19.17), the Fourier series can be ex- pressed in terms of exponential functions as (see Box 19.2 and App. A)

f(t) 5 a q

k52q c̃ke

ikv0t (19.21)

where i 5 121 and

c̃k 5 1

T #

Ty2

2Ty2 f(t)e2ikv0t dt (19.22)

This alternative formulation will have utility throughout the remainder of the chapter.

19.3 FREQUENCY AND TIME DOMAINS

To this point, our discussion of Fourier approximation has been limited to the time domain. We have done this because most of us are fairly comfortable conceptualizing a function’s behavior in this dimension. Although it is not as familiar, the frequency domain provides an alternative perspective for characterizing the behavior of oscillating functions. Thus, just as amplitude can be plotted versus time, so also can it be plotted versus frequency. Both types of expression are depicted in Fig. 19.7a, where we have drawn a three-dimensional graph of a sinusoidal function,

f(t) 5 C1 cos at 1 p2 b In this plot, the magnitude or amplitude of the curve, f(t), is the dependent variable and time t and frequency f 5 v0y2p are the independent variables. Thus, the amplitude and the time axes form a time plane, and the amplitude and the frequency axes form a

cha9792x_ch19_526-562.indd Page 536 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 536 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.3 FREQUENCY AND TIME DOMAINS 537

frequency plane. The sinusoid can, therefore, be conceived of as existing a distance 1yT out along the frequency axis and running parallel to the time axes. Consequently, when we speak about the behavior of the sinusoid in the time domain, we mean the projection of the curve onto the time plane (Fig. 19.7b). Similarly, the behavior in the frequency domain is merely its projection onto the frequency plane. As in Fig. 19.7c, this projection is a measure of the sinusoid’s maximum positive amplitude C1. The full peak-to-peak swing is unnecessary because of the symmetry. To- gether with the location 1yT along the frequency axis, Fig. 19.7c now defi nes the amplitude and frequency of the sinusoid. This is enough information to reproduce the shape and size of the curve in the time domain. However, one more parameter, namely, the phase angle, is required to position the curve relative to t 5 0. Consequently, a phase diagram, as shown in Fig. 19.7d, must also be included. The phase angle is determined as the distance (in radians) from zero to the point at which the positive peak occurs. If the peak occurs after

Box 19.2 Complex Form of the Fourier Series

The trigonometric form of the continuous Fourier series is

f (t) 5 a0 1 a q

k51 [ak cos(kv0t) 1 bk sin(kv0t) ] (B19.2.1)

From Euler’s identity, the sine and cosine can be expressed in expo- nential form as

sin x 5 eix 2 e2ix

2i (B19.2.2)

cos x 5 eix 1 e2ix

2 (B19.2.3)

which can be substituted into Eq. (B19.2.1) to give

f (t) 5 a0 1 a q

k51 aeikv0t ak 2 ibk

2 1 e2ikv0t

ak 1 ibk 2

b (B19.2.4) because 1yi 5 2i. We can defi ne a number of constants

c̃0 5 a0

c̃k 5 ak 2 ibk

2

c̃2k 5 a2k 2 ib2k

2 5

ak 1 ibk 2

(B19.2.5)

where, because of the odd and even properties of the sine and co- sine, ak 5 a2k and bk 5 2b2k. Equation (B19.2.4) can, therefore, be reexpressed as

f (t) 5 c̃0 1 a q

k51 c̃ke

ikv0t 1 a q

k51 c̃2ke

2ikv0t

or

f (t) 5 a q

k50 c̃ke

ikv0t 1 a q

k51 c̃2ke

2ikv0t

To simplify further, instead of summing the second series from 1 to `, perform the sum from 21 to 2`,

f (t) 5 a q

k50 c̃ke

ikv0t 1 a 2q

k521 c̃ke

ikv0t

or

f (t) 5 a q

k52q c̃ke

ikv0t (B19.2.6)

where the summation includes a term for k 5 0. To evaluate the c̃k’s, Eqs. (19.18) and (19.19) can be substituted into Eq. (B19.2.5) to yield

c̃k 5 1

T #

Ty2

2Ty2 f (t) cos(kv0t) dt 2 i

1

T #

Ty2

2Ty2 f(t) sin(kv0t) dt

Employing Eqs. (B19.2.2) and (B19.2.3) and simplifying gives

c̃k 5 1

T #

Ty2

2Ty2 f (t)e2ikv0t dt (B19.2.7)

Therefore, Eqs. (B19.2.6) and (B19.2.7) are the complex versions of Eqs. (19.17) through (19.20). Note that App. A includes a sum- mary of the interrelationships among all the formats of the Fourier series introduced in this chapter.

cha9792x_ch19_526-562.indd Page 537 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 537 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

538 FOURIER APPROXIMATION

zero, it is said to be delayed (recall our discussion of lags and leads in Sec. 19.1), and by convention, the phase angle is given a negative sign. Conversely, a peak before zero is said to be advanced and the phase angle is positive. Thus, for Fig. 19.7, the peak leads zero and the phase angle is plotted as 1py2. Figure 19.8 depicts some other possibilities. We can now see that Fig. 19.7c and d provides an alternative way to present or summarize the pertinent features of the sinusoid in Fig. 19.7a. They are referred to as line spectra. Admittedly, for a single sinusoid they are not very interesting. However, when applied to a more complicated situation, say, a Fourier series, their true power and value is revealed. For example, Fig. 19.9 shows the amplitude and phase line spectra for the square-wave function from Example 19.2. Such spectra provide information that would not be apparent from the time domain. This can be seen by contrasting Figs. 19.6 and 19.9. Figure 19.6 presents two alternative time-domain perspectives. The fi rst, the original square wave, tells us nothing about the

FIGURE 19.7 (a) A depiction of how a sinusoid can be portrayed in the time and the frequency domains. The time projection is reproduced in (b), whereas the amplitude-frequency projection is reproduced in (c). The phase-frequency projection is shown in (d ).

T//1

T

f (t)

f (t)

t

tf

C1 (b)

(a)

(c)

Frequency

Ti m e

0 0 1/T f f

C1

A m

p lit

u d

e

(d )

0 1/T

P h

as e

– �

cha9792x_ch19_526-562.indd Page 538 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 538 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

539

FIGURE 19.8 Various phases of a sinusoid showing the associated phase line spectra.

– �

– �

– �

– �

– �

FIGURE 19.9 (a) Amplitude and (b) phase line spectra for the square wave from Fig. 19.5.

4/�

2/�

3f0 5f0 7f0 (a)

f0 f

� /2

– �

– � /2

3f0 5f0 7f0

(b)

f0 f

cha9792x_ch19_526-562.indd Page 539 18/12/13 9:39 AM F-468 cha9792x_ch19_526-562.indd Page 539 18/12/13 9:39 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

540 FOURIER APPROXIMATION

sinusoids that comprise it. The alternative is to display these sinusoids—that is, (4yp) cos(v0t), 2(4y3p) cos(3v0t), (4y5p) cos(5v0t), etc. This alternative does not provide an adequate visualization of the structure of these harmonics. In contrast, Fig. 19.9a and b provides a graphic display of this structure. As such, the line spectra represent “fi nger- prints” that can help us to characterize and understand a complicated waveform. They are particularly valuable for nonidealized cases where they sometimes allow us to discern structure in otherwise obscure signals. In the next section, we will describe the Fourier transform that will allow us to extend such analyses to nonperiodic waveforms.

19.4 FOURIER INTEGRAL AND TRANSFORM

Although the Fourier series is a useful tool for investigating the spectrum of a periodic function, there are many waveforms that do not repeat themselves regularly. For example, a lightning bolt occurs only once (or at least it will be a long time until it occurs again), but it will cause interference with receivers operating on a broad range of frequencies— for example, TVs, radios, and shortwave receivers. Such evidence suggests that a non- recurring signal such as that produced by lightning exhibits a continuous frequency spectrum. Because such phenomena are of great interest to engineers, an alternative to the Fourier series would be valuable for analyzing these aperiodic waveforms. The Fourier integral is the primary tool available for this purpose. It can be derived from the exponential form of the Fourier series

f(t) 5 a q

k52q c̃ke

ikv0t (19.23)

where

c̃k 5 1

T #

Ty2

2Ty2 f(t)e2ikv0t dt (19.24)

where v0 5 2pyT and k 5 0, 1, 2, . . . . The transition from a periodic to a nonperiodic function can be effected by allowing the period to approach infi nity. In other words, as T becomes infi nite, the function never repeats itself and thus becomes aperiodic. If this is allowed to occur, it can be demon- strated (for example, Van Valkenburg, 1974; Hayt and Kemmerly, 1986) that the Fourier series reduces to

f(t) 5 1

2p # q

2q F(iv0)e

iv0t dv0 (19.25)

and the coeffi cients become a continuous function of the frequency variable v, as in

F(iv0) 5 # q

2q f(t)e2iv0t dt (19.26)

The function F(iv0), as defi ned by Eq. (19.26), is called the Fourier integral of f(t). In addition, Eqs. (19.25) and (19.26) are collectively referred to as the Fourier transform pair. Thus, along with being called the Fourier integral, F(iv0) is also called the Fourier transform of f(t). In the same spirit, f(t), as defi ned by Eq. (19.25), is referred to as the

cha9792x_ch19_526-562.indd Page 540 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 540 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.4 FOURIER INTEGRAL AND TRANSFORM 541

inverse Fourier transform of F(iv0). Thus, the pair allows us to transform back and forth between the time and the frequency domains for an aperiodic signal. The distinction between the Fourier series and transform should now be quite clear. The major difference is that each applies to a different class of functions—the series to periodic and the transform to nonperiodic waveforms. Beyond this major distinction, the two approaches differ in how they move between the time and the frequency domains. The Fourier series converts a continuous, periodic time-domain function to frequency- domain magnitudes at discrete frequencies. In contrast, the Fourier transform converts a continuous time-domain function to a continuous frequency-domain function. Thus, the discrete frequency spectrum generated by the Fourier series is analogous to a continuous frequency spectrum generated by the Fourier transform. The shift from a discrete to a continuous spectrum can be illustrated graphically. In Fig. 19.10a, we can see a pulse train of rectangular waves with pulse widths equal to one-half the period along with its associated discrete spectrum. This is the same function as was investigated previously in Example 19.2, with the exception that it is shifted vertically.

�t

�t

t0

(a)

(b)

T

t0 T

(c)

t

f

f

f0 T

FIGURE 19.10 Illustration of how the discrete frequency spectrum of a Fourier series for a pulse train (a) approaches a continuous frequency spectrum of a Fourier integral (c) as the period is allowed to approach infi nity.

cha9792x_ch19_526-562.indd Page 541 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 541 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

542 FOURIER APPROXIMATION

In Fig. 19.10b, a doubling of the pulse train’s period has two effects on the spectrum. First, two additional frequency lines are added on either side of the original components. Second, the amplitudes of the components are reduced. As the period is allowed to approach infi nity, these effects continue as more and more spectral lines are packed together until the spacing between lines goes to zero. At the limit, the series converges on the continuous Fourier integral, depicted in Fig.  19.10c. Now that we have introduced a way to analyze an aperiodic signal, we will take the fi nal step in our development. In the next section, we will acknowledge the fact that a signal is rarely characterized as a continuous function of the sort needed to implement Eq. (19.26). Rather, these data are invariably in a discrete form. Thus, we will now show how to compute a Fourier transform for such discrete measurements.

19.5 DISCRETE FOURIER TRANSFORM (DFT)

In engineering, functions are often represented by fi nite sets of discrete values. Addi- tionally, data are often collected in or converted to such a discrete format. As depicted in Fig. 19.11, an interval from 0 to t can be divided into N equispaced subintervals with widths of Dt 5 TyN. The subscript n is employed to designate the discrete times at which samples are taken. Thus, fn designates a value of the continuous function f(t) taken at tn.

f (t)

t0 t1 t2

f3f2 f1

f0

tn = Ttn – 1

fn – 1

FIGURE 19.11 The sampling points of the discrete Fourier series.

cha9792x_ch19_526-562.indd Page 542 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 542 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.5 DISCRETE FOURIER TRANSFORM (DFT) 543

Note that the data points are specifi ed at n 5 0, 1, 2, . . . , N 2 1. A value is not included at n 5 N. (See Ramirez, 1985, for the rationale for excluding fN.) For the system in Fig. 19.11, a discrete Fourier transform can be written as

Fk 5 a N21

n50 fne

2ikv0n  for k 5 0 to N 2 1 (19.27)

and the inverse Fourier transform as

fn 5 1

N a N21

k50 Fke

ikv0n  for n 5 0 to N 2 1 (19.28)

where v0 5 2pyN. Equations (19.27) and (19.28) represent the discrete analogs of Eqs. (19.26) and (19.25), respectively. As such, they can be employed to compute both a direct and an inverse Fourier transform for discrete data. Although such calculations can be performed by hand, they are extremely arduous. As expressed by Eq. (19.27), the DFT requires N2 complex operations. Thus, we will now develop a computer algorithm to implement the DFT.

Computer Algorithm for the DFT. Note that the factor 1yN in Eq. (19.28) is merely a scale factor that can be included in either Eq. (19.27) or (19.28), but not both. For our computer algorithm, we will shift it to Eq. (19.27) so that the fi rst coeffi cient F0 (which is the analog of the continuous coeffi cient a0) is equal to the arithmetic mean of the samples. Also, to develop an algorithm that can be implemented in languages that do not accommodate complex variables, we can use Euler’s identity,

e6ia 5 cos a 6 i sin a

to reexpress Eqs. (19.27) and (19.28) as

Fk 5 1

N a N

n50 [

  fn cos(kv0n) 2 ifn sin(kv0n)] (19.29)

and

fn 5 a N21

k50 [Fk cos(kv0n) 2 iFk sin(kv0n)] (19.30)

Pseudocode to implement Eq. (19.29) is listed in Fig. 19.12. This algorithm can be developed into a computer program to compute the DFT. The output from such a program is listed in Fig. 19.13 for the analysis of a cosine function.

cha9792x_ch19_526-562.indd Page 543 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 543 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

544 FOURIER APPROXIMATION

19.6 FAST FOURIER TRANSFORM (FFT)

Although the algorithm described in the previous section adequately calculates the DFT, it is computationally burdensome because N2 operations are required. Consequently, for data samples of even moderate size, the direct determination of the DFT can be extremely time-consuming. The fast Fourier transform, or FFT, is an algorithm that has been developed to compute the DFT in an extremely economical fashion. Its speed stems from the fact that it utilizes the results of previous computations to reduce the number of operations. In particular, it exploits the periodicity and symmetry of trigonometric functions to compute

FIGURE 19.12 Pseudocode for computing the DFT.

DOFOR k 5 0, N 2 1 DOFOR n 5 0, N 2 1 angle 5 kv0n realk 5 realk 1 fn cos(angle)/N imaginaryk 5 imaginaryk 2 fn sin(angle)/N END DO END DO

FIGURE 19.13 Output of a program based on the algorithm from Fig. 19.12 for the DFT of data generated by a cosine function f(t) 5 cos[2p(12.5)t] at 16 points with Dt 5 0.01 s.

INDEX f(t) REAL IMAGINARY 0 1.000 0.000 0.000 1 0.707 0.000 0.000 2 0.000 0.500 0.000 3 20.707 0.000 0.000 4 21.000 0.000 0.000 5 20.707 0.000 0.000 6 0.000 0.000 0.000 7 0.707 0.000 0.000 8 1.000 0.000 0.000 9 0.707 0.000 0.000 10 0.000 0.000 0.000 11 20.707 0.000 0.000 12 21.000 0.000 0.000 13 20.707 0.000 0.000 14 0.000 0.500 0.000 15 0.707 0.000 0.000

cha9792x_ch19_526-562.indd Page 544 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 544 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.6 FAST FOURIER TRANSFORM (FFT) 545

the transform with approximately N log2 N operations (Fig. 19.14). Thus, for N 5 50 samples, the FFT is about 10 times faster than the standard DFT. For N 5 1000, it is about 100 times faster. The fi rst FFT algorithm was developed by Gauss in the early nineteenth century (Heideman et al., 1984). Other major contributions were made by Runge, Danielson, Lanczos, and others in the early twentieth century. However, because discrete transforms often took days to weeks to calculate by hand, they did not attract broad interest prior to the development of the modern digital computer. In 1965, J. W. Cooley and J. W. Tukey published a key paper in which they outlined an algorithm for calculating the FFT. This scheme, which is similar to those of Gauss and other earlier investigators, is called the Cooley-Tukey algorithm. Today, there are a host of other approaches that are offshoots of this method. The basic idea behind each of these algorithms is that a DFT of length N is decom- posed, or “decimated,” into successively smaller DFTs. There are a variety of different ways to implement this principle. For example, the Cooley-Tukey algorithm is a member of what are called decimation-in-time techniques. In the present section, we will describe an alternative approach called the Sande-Tukey algorithm. This method is a member of another class of algorithms called decimation-in-frequency techniques. The distinction between the two classes will be discussed after we have elaborated on the method.

0

DFT(�N2)

40 Samples

1000

2000

O p

er at

io n

s

FFT(� N log2

N)

FIGURE 19.14 Plot of number of operations vs. sample size for the standard DFT and the FFT.

cha9792x_ch19_526-562.indd Page 545 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 545 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

546 FOURIER APPROXIMATION

19.6.1 Sande-Tukey Algorithm

In the present case, N will be assumed to be an integral power of 2,

N 5 2M (19.31)

where M is an integer. This constraint is introduced to simplify the resulting algorithm. Now, recall that the DFT can be generally represented as

Fk 5 a N21

n50 fne

2i(2pyN)nk  for k 5 0 to N 2 1 (19.32)

where 2pyN 5 v0. Equation (19.32) can also be expressed as

Fk 5 a N21

n50 fnW

nk

where W is a complex-valued weighting function defi ned as

W 5 e2i(2pyN) (19.33)

Suppose now that we divide the sample in half and express Eq. (19.32) in terms of the fi rst and last Ny2 points:

Fk 5 a (Ny2)21

n50 fne

2i(2pyN)kn 1 a N21

n5Ny2 fne

2i(2pyN)kn

where k 5 0, 1, 2, . . . , N 2 1. A new variable, m 5 n 2 Ny2, can be created so that the range of the second summation is consistent with the fi rst,

Fk 5 a (Ny2)21

n50 fne

2i(2pyN)kn 1 a (Ny2)21

m50 fm1Ny2e

2i(2pyN)k(m1Ny2)

or

Fk 5 a (Ny2)21

n50 ( fn 1 e

2ipkfn1Ny2)e 2i2pknyN (19.34)

Next, recognize that the factor e2ipk 5 (21)k. Thus, for even points it is equal to 1 and for odd points it is equal to 21. Therefore, the next step in the method is to separate Eq. (19.34) according to even values and odd values of k. For the even values,

F2k 5 a (Ny2)21

n50 ( fn 1 fn1Ny2)e

2i2p(2k)nyN 5 a (Ny2)21

n50 ( fn 1 fn1Ny2)e

2i2pkny(Ny2)

and for the odd values,

F2k11 5 a (Ny2)21

n50 ( fn 2 fn1Ny2)e

2i2p(2k11)nyN

5 a (Ny2)21

n50 ( fn 2 fn1Ny2)e

2i2pnyNe2i2pkny(Ny2)

for k 5 0, 1, 2, . . . , (Ny2) 2 1.

cha9792x_ch19_526-562.indd Page 546 30/10/13 8:24 PM f-512 cha9792x_ch19_526-562.indd Page 546 30/10/13 8:24 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.6 FAST FOURIER TRANSFORM (FFT) 547

These equations can also be expressed in terms of Eq. (19.33). For the even values,

F2k 5 a (Ny2)21

n50 ( fn 1 fn1Ny2)W

2kn

and for the odd values,

F2k11 5 a (Ny2)21

n50 ( fn 2 fn1Ny2)W

nW 2kn

Now, a key insight can be made. These even and odd expressions can be interpreted as being equal to the transforms of the (Ny2)-length sequences

gn 5 fn 1 fn1Ny2 (19.35)

and

hn 5 ( fn 2 fn1Ny2)W n  for n 5 0, 1, 2, p , (Ny2) 2 1 (19.36)

Thus, it directly follows that

F2k 5 Gk F2k11 5 Hk

f  for k 5 0, 1, 2, p , (Ny2) 2 1 In other words, one N-point computation has been replaced by two (Ny2)-point computations. Because each of the latter requires approximately (Ny2)2 complex multi- plications and additions, the approach produces a factor-of-2 savings—that is, N2 versus 2(Ny2)2 5 N2y2. The scheme is depicted in Fig. 19.15 for N 5 8. The DFT is computed by fi rst forming the sequence gn and hn and then computing the Ny2 DFTs to obtain the even- and odd-numbered transforms. The weights Wn are sometimes called twiddle factors. Now it is clear that this “divide-and-conquer” approach can be repeated at the sec- ond stage. Thus, we can compute the (Ny4)-point DFTs of the four Ny4 sequences composed of the fi rst and last Ny4 points of Eqs. (19.35) and (19.36). The strategy is continued to its inevitable conclusion when Ny2 two-point DFTs are computed (Fig. 19.16). The total number of calculations for the entire computation is on the order of N log2 N. The contrast between this level of effort and that of the standard DFT (Fig. 19.14) illustrates why the FFT is so important.

Computer Algorithm. It is a relatively straightforward proposition to express Fig. 19.16 as an algorithm. As was the case for the DFT algorithm of Fig. 19.12, we will use Euler’s identity,

e6ia 5 cos a 6 i sin a

to allow the algorithm to be implemented in languages that do not explicitly accommo- date complex variables. Close inspection of Fig. 19.16 indicates that its fundamental computational molecule is the so-called butterfl y network depicted in Fig. 19.17a. Pseudocode to implement one of these molecules is shown in Fig. 19.17b.

cha9792x_ch19_526-562.indd Page 547 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 547 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

548 FOURIER APPROXIMATION

f (5)

f (4)

f (7)

f (6)

f (3)

f (2)

f (1)

f (0)

F (5)

F (1)

F (7)

F (3)

F (6)

F (2)

F (4)

F (0)

W 0

W 1

W 2

W 3

W 0

W 2

W 0

W 0

W 0

W 0

W 0

W 2

+

+ +

+ +

+ +

+ +

– +

– +

– +

+

+ +

+ +

– +

– +

+ +

+ +

– +

+

+ +

– +

+ +

– +

+ +

– +

+ +

FIGURE 19.16 Flow graph of the complete decimation-in-frequency decomposition of an eight-point DFT.

(N/2)-point DFT

(N/2)-point DFT

f (5)

f (4)

f (7)

f (6)

f (3)

f (2)

f (1)

f (0)

F (3)

F (1)

F (7)

F (5)

F (6)

F (4)

F (2)

F (0)

W 0

W 1

W 2

W 3 h(3)

h(2)

h(1)

h(0)

g(3)

g(2)

g(1)

g(0)+

+ +

+ +

+ +

+ +

– +

– +

– +

FIGURE 19.15 Flow graph of the fi rst stage in a decimation-in-frequency decomposition of an N-point DFT into two (Ny2)-point DFTs for N 5 8.

cha9792x_ch19_526-562.indd Page 548 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 548 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.6 FAST FOURIER TRANSFORM (FFT) 549

FIGURE 19.17 (a) A butterfl y network that represents the fundamental computation of Fig. 19.16. (b) Pseudocode to implement (a).

f (0)

f (1)

F(0)

F(1)

+

+

+

– (a) (b)

temporary real (1) real (0) temporary imaginary (1) imaginary (0)

= real (0) + real (1) = real (0) – real (1) = temporary = imaginary (0) + imaginary (1) = imaginary (0) – imaginary (1) = temporary

FIGURE 19.18 Pseudocode to implement a decimation-in-frequency FFT. Note that the pseudocode is composed of two parts: (a) the FFT itself and (b) a bit-reversal routine to unscramble the order of the resulting Fourier coeffi cients.

(a) m 5 LOG(N)/LOG(2) N2 5 N DOFOR k 5 1, m N1 5 N2 N2 5 N2/2 angle 5 0 arg 5 2p/N1 DOFOR j 5 0, N2 2 1 c 5 cos(angle) s 5 2sin(angle) DOFOR i 5 j, N 2 1, N1 kk 5 i 1 N2 xt 5 x(i) 2 x(kk) x(i) 5 x(i) 1 x(kk) yt 5 y(i) 2 y(kk) y(i) 5 y(i) 1 y(kk) x(kk) 5 xt * c 2 yt * s y(kk) 5 yt * c 1 xt * s END DO angle 5 (j 1 1) * arg END DO END DO

(b) j 5 0 DOFOR i 5 0, N 2 2 IF (i , J) THEN xt 5 xj xj 5 xi xi 5 xt yt 5 yj yj 5 yi yi 5 yt END IF k 5 N/2 DO IF (k $ j 1 1) EXIT j 5 j 2 k k 5 k/2 END DO j 5 j 1 k END DO DOFOR i 5 0, N 2 1 x(i) 5 x(i)/N y(i) 5 y(i)/N END DO

Pseudocode for the FFT is listed in Fig. 19.18. The fi rst part consists essentially of three nested loops to implement the computation embodied in Fig. 19.16. Note that the real-valued data are originally stored in the array x. Also note that the outer loop steps through the M stages [recall Eq. (19.31)] of the fl ow graph. After this fi rst part is executed, the DFT will have been computed but in a scrambled order (see the right-hand side of Fig. 19.16). These Fourier coeffi cients can be unscrambled

cha9792x_ch19_526-562.indd Page 549 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 549 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

550 FOURIER APPROXIMATION

by a procedure called bit reversal. If the subscripts 0 through 7 are expressed in binary, the correct ordering can be obtained by reversing these bits (Fig. 19.19). The second part of the algorithm implements this procedure.

19.6.2 Cooley-Tukey Algorithm

Figure 19.20 shows a fl ow network to implement the Cooley-Tukey algorithm. For this case, the sample is initially divided into odd- and even-numbered points, and the fi nal results are in correct order.

FIGURE 19.19 Depiction of the bit-reversal process.

Scrambled Scrambled Bit-Reversed Final Order Order Order Result (Decimal) (Binary) (Binary) (Decimal)

F(0) F(000) F(000) F(0) F(4) F(100) F(001) F(1) F(2) F(010) F(010) F(2) F(6) ⇒ F(110) ⇒ F(011) ⇒ F(3) F(1) F(001) F(100) F(4) F(5) F(101) F(101) F(5) F(3) F(011) F(110) F(6) F(7) F(111) F(111) F(7)

FIGURE 19.20 Flow graph of a decimation-in-time FFT of an eight-point DFT.

f (5)

f (1)

f (7)

f (3)

f (6)

f (2)

f (4)

f (0)

F (5)

F (4)

F (7)

F (6)

F (3)

F (2)

F (1)

F (0)

W0

W0

W0

W0

W2

W2

W0

W0

W3

W2

W1

W0

cha9792x_ch19_526-562.indd Page 550 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 550 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.7 THE POWER SPECTRUM 551

This approach is called a decimation in time. It is the reverse of the Sande-Tukey algorithm described in the previous section. Although the two classes of method differ in organization, they both exhibit the N log2 N operations, which are the strength of the FFT approach.

19.7 THE POWER SPECTRUM

The FFT has many engineering applications, ranging from vibration analysis of structures and mechanisms to signal processing. As described previously, amplitude and phase spectra provide a means to discern the underlying structure of seemingly random signals. Similarly, a useful analysis called a power spectrum can be developed from the Fourier transform. As the name implies, the power spectrum derives from the analysis of the power output of electrical systems. In mathematical terms, the power of a periodic signal in the time domain can be defi ned as

P 5 1

T #

Ty2

2Ty2 f 2(t) dt (19.37)

Now another way to look at this information is to express it in the frequency domain by calculating the power associated with each frequency component. This information can be then displayed as a power spectrum, a plot of the power versus frequency. If the Fourier series for f(t) is

f(t) 5 a q

k52q Fke

ikv0t (19.38)

the following relation holds (see Gabel and Roberts, 1987, for details):

1

T #

Ty2

2Ty2 f 2(t) dt 5 a

q

k52q ZFk Z

2 (19.39)

Thus, the power in f(t) can be determined by adding together the squares of the Fourier coefficients, that is, the powers associated with the individual frequency components. Now, remember that in this representation, the single real harmonic consists of both frequency components at 6kv0. We also know that the positive and nega- tive coefficients are equal. Therefore, the power in fk(t), the kth real harmonic of f (t), is

pk 5 2 ZFk Z 2 (19.40)

The power spectrum is the plot of pk as a function of frequency kv0. We will devote Sec. 20.3 to an engineering application involving the FFT and the power spectrum generated with software packages.

cha9792x_ch19_526-562.indd Page 551 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 551 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

552 FOURIER APPROXIMATION

Additional Information. The foregoing has been a brief introduction to Fourier ap- proximation and the FFT. Additional information on the former can be found in Van Valkenburg (1974), Chirlian (1969), and Hayt and Kemmerly (1986). References on the FFT include Davis and Rabinowitz (1975); Cooley, Lewis, and Welch (1977); and Brigham (1974). Nice introductions to both can be found in Ramirez (1985), Oppenheim and Schafer (1975), and Gabel and Roberts (1987).

19.8 CURVE FITTING WITH SOFTWARE PACKAGES

Software packages have great capabilities for curve fi tting. In this section, we will give you a taste of some of the more useful ones.

19.8.1 Excel

In the present context, the most useful application of Excel is for regression analysis and, to a lesser extent, polynomial interpolation. Aside from a few built-in functions (see Table 19.1), there are two primary ways in which this capability can be implemented: the Trendline tool and the Data Analysis ToolPak.

The Trendline Tool. This tool allows a number of different trend models to be added to a chart. These models include linear, polynomial, logarithmic, exponential, power, and moving average fi ts. The following example illustrates how the Trendline command is invoked.

TABLE 19.1 Excel built-in functions related to regression fi ts of data.

Function Description

FORECAST Returns a value along a linear trend GROWTH Returns values along an exponential trend INTERCEPT Returns the intercept of the linear regression line LINEST Returns the parameters of a linear trend LOGEST Returns the parameters of an exponential trend SLOPE Returns the slope of the linear regression line TREND Returns values along a linear trend

S O F T W A R E

EXAMPLE 19.3 Using Excel’s Trendline Command

Problem Statement. You may have noticed that several of the fi ts available on Trendline were discussed previously in Chap. 17 (for example, linear, polynomial, exponential, and power). An additional capability is the logarithmic model

y 5 a0 1 a1 log x

cha9792x_ch19_526-562.indd Page 552 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 552 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.8 CURVE FITTING WITH SOFTWARE PACKAGES 553

Fit the following data with this model using Excel’s Trendline command:

x 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5

y 0.53 0.69 1.5 1.5 2 2.06 2.28 2.23 2.73 2.42 2.79

Solution. To invoke the Trendline tool, a chart relating a series of dependent and independent variables must be created. For the present case, we fi rst create an XY-plot of the data. Next, we can select the chart (by clicking on it) and the series (by positioning the mouse arrow on one of the values and right clicking). A menu will appear from which you can select Add Trendline. At this point, a Format Trendline box opens where you can select the Trend/Regression Type as Logarithmic. In addition, select Display Equation on chart and Display R-squared value on chart. When the box is closed, the resulting fi t along with r2 is displayed as in Fig. 19.21. The Trendline command provides a handy way to fi t a number of commonly used models to data. In addition, its inclusion of the Polynomial option means that it can also be used for polynomial interpolation. However, the fact that its statistical content is limited to r2 means that it does not allow statistical inferences to be drawn regarding the model fi t. The Data Analysis ToolPak described next provides a nice alternative where such inferences are necessary.

3y

y = 0.9846 Ln (x) + 1.0004 r2 = 0.9444

0

1

2

0 2 4 6 x

FIGURE 19.21 Fit of a logarithmic model to the data from Example 19.3.

The Data Analysis ToolPak. This Excel Add-In Package contains a comprehensive capability for curve fi tting with general linear least squares. As previously described in Sec. 17.4, such models are of the general form

y 5 a0z0 1 a1z1 1 a2z2 1 p 1 amzm 1 e (17.23)

where z0, z1, . . . , zm are m 1 1 different functions. The next example illustrates how such models can be fi t with Excel.

cha9792x_ch19_526-562.indd Page 553 30/10/13 9:38 PM f-512 cha9792x_ch19_526-562.indd Page 553 30/10/13 9:38 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

554 FOURIER APPROXIMATION

EXAMPLE 19.4 Using Excel’s Data Analysis ToolPak

Problem Statement. The following data were collected for the slope, hydraulic radius, and velocity of water fl owing in a canal:

S, m/m 0.0002 0.0002 0.0005 0.0005 0.001 0.001

R, m 0.2 0.5 0.2 0.5 0.2 0.5

U, m/s 0.25 0.5 0.4 0.75 0.5 1

There are theoretical reasons for believing that these data can be fi t to a power model of the form

U 5 aSs Rr

where a, s, and r are empirically derived coeffi cients. There are also theoretical reasons for believing that s and r should have values of approximately 0.5 and 0.667, respec- tively. Fit these data with Excel and evaluate whether your regression estimates contradict the expected values for the model coeffi cients.

Solution. The logarithm of the power model is fi rst used to convert it to the linear format of Eq. (17.23),

U 5 log a 1 s log S 1 r log R

An Excel spreadsheet can be developed with both the original data along with their common logarithms, as in the following:

As shown, an effi cient way to generate the logarithms is to type the formula to compute the fi rst log(S). This formula can then be copied to the right and down to generate the other logarithms. Because of its status as an “add-in” on the version of Excel available at the time of this book’s printing, the Data Analysis ToolPak must sometimes be loaded into Excel. To do this, choose File, Options, Add-Ins, Manage Excel Add-ins. Then, check the Anal- ysis ToolPak box. The ToolPak then should be installed and a button to access it should appear on your Data tab.

S O F T W A R E

cha9792x_ch19_526-562.indd Page 554 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 554 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.8 CURVE FITTING WITH SOFTWARE PACKAGES 555

After selecting Data Analysis from the Data menu, a Data Analysis menu will ap- pear on the screen containing a large number of statistically oriented routines. Select Regression and a dialogue box will appear, prompting you for information on the regres- sion. After making sure that the default selection New Worksheet Ply is selected, fi ll in F2:F7 for the y range and D2:E7 for the x range, and select OK. The following worksheet will be created:

Thus, the resulting fi t is

log U 5 1.522 1 0.433 log S 1 0.733 log R

or by taking the antilog,

U 5 33.3S0.433 R0.733

Notice that 95% confi dence intervals are generated for the coeffi cients. Thus, there is a 95% probability that the true slope exponent falls between 0.363 and 0.504, and the true hydraulic radius coeffi cient falls between 0.631 and 0.835. Thus, the fi t does not contradict the theoretical exponents.

Finally, it should be noted that the Excel Solver tool can be used to perform nonlinear regression by directly minimizing the sum of the squares of the residuals between a nonlin- ear model prediction and data. We devote Sec. 20.1 to an example of how this can be done.

19.8.2 MATLAB

As summarized in Table 19.2, MATLAB software has a variety of built-in functions that span the total capabilities described in this part of the book. The following example illustrates how a few of them can be used.

cha9792x_ch19_526-562.indd Page 555 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 555 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

556 FOURIER APPROXIMATION

EXAMPLE 19.5 Using MATLAB for Curve Fitting

Problem Statement. Explore how MATLAB can be employed to fi t curves to data. To do this, use the sine function to generate equally spaced f(x) values from 0 to 10. Employ a step size of 1 so that the resulting characterization of the sine wave is sparse (Fig. 19.22). Then, fi t it with (a) linear interpolation, (b) a fi fth-order polynomial, and (c) a cubic spline.

Solution.

(a) The values for the independent and the dependent variables can be entered into vectors by

>> x=0:10; >> y=sin(x);

A new, more fi nely spaced vector of independent variable values can be generated and stored in the vector xi,

>> xi=0:.25:10;

The MATLAB function interp1 can then be used to generate dependent variable values yi for all the xi values using linear interpolation. Both the original data (x, y) along with the linearly interpolated values can be plotted together, as shown in the

TABLE 19.2 Some of the MATLAB functions to implement interpolation, regression, splines, and the FFT.

Function Description

polyfit Fit polynomial to data interp1 1-D interpolation (1-D table lookup) interp2 2-D interpolation (2-D table lookup) spline Cubic spline data interpolation fft Discrete Fourier transform

FIGURE 19.22 Eleven points sampled from a sinusoid.

y

1

0

–1

5 10 x

S O F T W A R E

cha9792x_ch19_526-562.indd Page 556 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 556 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.8 CURVE FITTING WITH SOFTWARE PACKAGES 557

graph below:

>> yi=interp1(x,y,xi); >> plot(x,y,’o’,xi,yi)

(b) Next, the MATLAB polyfit function can be used to generate the coeffi cients of a fi fth-order polynomial fi t of the original sparse data,

>> p=polyfit(x,y,5) p= 0.0008 –0.0290 0.3542 –1.6854 2.5860 –0.0915

where the vector p holds the polynomial’s coeffi cients. These can, in turn, be used to generate a new set of yi values, which can again be plotted along with the original sparse sample,

>> yi = polyval(p,xi); >> plot(x,y,’o’,xi,yi)

Thus, the polynomial captures the general pattern of the data, but misses most of the points.

(c) Finally, the MATLAB spline function can be used to fi t a cubic spline to the original sparse data in the form of a new set of yi values, which can again be plotted along with the original sparse sample,

cha9792x_ch19_526-562.indd Page 557 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 557 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

558 FOURIER APPROXIMATION

>> yi=spline(x,y,xi); >> plot(x,y,'o',xi,yi)

It should be noted that MATLAB also has excellent capabilities to perform Fourier analysis. We devote Sec. 20.3 to an example of how this can be done.

MATLAB has two built-in functions for two- and three-dimensional piecewise in- terpolation: interp2 and interp3. As you might expect from their names, these functions operate in a similar fashion to interp1. For example, a simple representation of the syntax of interp2 is

zi = interp2(x, y, z, xi, yi, 'method')

where x and y 5 matrices containing the coordinates of the points at which the values in the matrix z are given, zi 5 a matrix containing the results of the interpolation as evalu- ated at the points in the matrices xi and yi, and method 5 the desired method. Note that the methods are identical to those used by interp1; that is, linear, nearest, spline, and cubic. As with interp1, if the method argument is omitted, the default is linear interpolation. For example, interp2 can be used to make the same evaluation as in Example 18.11 as

>> x=[2 9]; >> y=[1 6]; >> z=[60 57.5;55 70]; >> interp2(x,y,z,5.25,4.8)

ans = 61.2143

19.8.3 Mathcad

Mathcad can perform a wide variety of statistical, curve-fi tting, and data-smoothing tasks. These include relatively simple jobs like plotting histograms and calculating population statistic summaries such as mean, median, variance, standard deviations, and correlation coeffi cients. In addition, Mathcad contains a number of functions for performing regres- sion. The slope and intercept functions return the slope and intercept of the least-squares

S O F T W A R E

cha9792x_ch19_526-562.indd Page 558 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 558 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

19.8 CURVE FITTING WITH SOFTWARE PACKAGES 559

regression fi t line. The regress function is used for nth-order polynomial regression of a complete data set. The loess function performs localized nth-order polynomial regression over spans of the data that you can specify. The interp function can be used to return intermediate values of y from a regression fi t for a given x point. The regress and loess functions can also perform multivariate polynomial regression. Mathcad also provides the linfi t function that is used to model data with a linear combination of arbitrary functions. Finally, the genfi t function is available for cases where model coeffi cients appear in arbi- trary form. In this case, the more diffi cult nonlinear equations must be solved by iteration. Mathcad also has considerable capabilities for interpolation. It can predict interme- diate values by connecting known data points with either straight lines (linear interpola- tion) using the linterp function or with cubic spline interpolation using cspline, pspline, or lspline. These spline functions allow you to try different ways to deal with interpolation at the end points of the data. The lspline function generates a spline curve that is a straight line at the end points. The pspline function generates a spline curve that is a parabola at the end points. The cspline function generates a spline curve that is cubic at the end points. The interp function uses the curve-fi tting results and returns an interpolated y  value given an x value. In addition, you can perform two-dimensional cubic spline interpolation by passing a surface through a grid of points. Let’s do an example that shows how Mathcad is used to perform spline interpolation (Fig. 19.23). The data we will fi t are simply some evenly spaced points sampled from a

FIGURE 19.23 Cubic spline interpolation with Mathcad.

cha9792x_ch19_526-562.indd Page 559 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 559 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

560 FOURIER APPROXIMATION

sinusoid. After generating these data, the defi nition symbol and the lspline function are used to compute the spline coeffi cients. Then, an interpolation function, fi t, is developed with the interp function in order to generate interpolated values for specifi c values of x. Mathcad designed this sequence of operations so that the interpolating polynomials would not have to be recalculated every time an interpolation is desired. With the functions in place, you can then interpolate at any location using fi t(x), as shown with x 5 2.5. You can also con- struct a plot of these data along with the interpolated spline as shown in Fig. 19.23. As another example of demonstrating some of Mathcad’s curve fi tting capabilities let’s use the fft function for Fourier analysis as in Fig. 19.24. The fi rst line uses the defi nition symbol to create i as a range variable. Next, xi is formulated using the rnd Mathcad func- tion to impart a random component to a sinusoidal signal. The graph of the signal can be placed on the worksheet by clicking to the desired location. This places a red crosshair at that location. Then use the Insert/Graph/X-Y Plot pull-down menu to place an empty plot on the worksheet with placeholders for the expressions to be graphed and for the ranges of the x and y axes. Simply type xi in the placeholder on the y axis and i for the x-axis range. Mathcad does the rest to produce the fi rst graph shown in Fig. 19.24. Once the graph has been created you can use the Format/Graph/X-Y Plot pull-down menu to vary the type of graph; change the color, type, and weight of the trace of the function; and add titles, labels, and other features. Next, c is defi ned as fft(x). This function returns the Fourier transform of x. The result is a vector c of complex coeffi cients that represent values in the frequency domain. A plot of the magnitude of cj is then constructed as described above.

FIGURE 19.24 FFT with Mathcad.

S O F T W A R E

cha9792x_ch19_526-562.indd Page 560 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 560 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 561

PROBLEMS

19.1 The average values of a function can be determined by

f (x) 5 ex0 f (x)dx

x

Use this relationship to verify the results of Eq. (19.13). 19.2 The solar radiation for Tucson, Arizona, has been tabulated as

Time, mo J F M A M J J A S O N D

Radiation, W/m2 144 188 245 311 351 359 308 287 260 211 159 131

19.4 Use a continuous Fourier series to approximate the sawtooth wave in Fig. P19.4. Plot the fi rst three terms along with the sum- mation. 19.5 Use a continuous Fourier series to approximate the wave form in Fig. P19.5. Plot the fi rst three terms along with the sum- mation.

Assuming each month is 30 days long, fi t a sinusoid to these data. Use the resulting equation to predict the radiation in mid-August. 19.3 The pH in a reactor varies sinusoidally over the course of a day. Use least-squares regression to fi t Eq. (19.11) to the following data. Use your fi t to determine the mean, amplitude, and time of maximum pH. Note that the period is 24 hr.

Time, hr 0 2 4 5 7 9 12 15 20 22 24

pH 7.6 7 7.1 6.5 7.4 7.2 8.9 8.8 8.9 7.9 7

FIGURE P19.4 A sawtooth wave.

1

T t

– 1 – 1

FIGURE P19.5 A triangular wave.

t2

1

–2

19.6 Construct amplitude and phase line spectra for Prob. 19.4. 19.7 Construct amplitude and phase line spectra for Prob. 19.5. 19.8 A half-wave rectifi er can be characterized by

C1 5 c 1 p

1 1

2 sin t 2

2

3p cos 2t 2

2

15p cos 4t

2 2

35p cos 6t 2 p d

where C1 is the amplitude of the wave. Plot the fi rst four terms along with the summation. 19.9 Construct amplitude and phase line spectra for Prob. 19.8. 19.10 Develop a user-friendly program for the DFT based on the algorithm from Fig. 19.12. Test it by duplicating Fig. 19.13. 19.11 Use the program from Prob. 19.10 to compute a DFT for the triangular wave from Prob. 19.8. Sample the wave from t 5 0 to 4T. Use 32, 64, and 128 sample points. Time each run and plot execu- tion versus N to verify Fig. 19.14. 19.12 Develop a user-friendly program for the FFT based on the algorithm from Fig. 19.18. Test it by duplicating Fig. 19.13. 19.13 Repeat Prob. 19.11 using the software you developed in Prob. 19.12. 19.14 An object is suspended in a wind tunnel and the force mea- sured for various levels of wind velocity. The results are tabulated below. Use Excel’s Trendline command to fi t a power equation to these data. Plot F versus y along with the power equation and r2.

v, m/s 10 20 30 40 50 60 70 80

F, N 25 70 380 550 610 1220 830 1450

19.15 Use the Excel Data Analysis ToolPak to develop a regression polynomial to the following data for the dissolved oxygen concentra- tion of fresh water versus temperature at sea level. Determine the order of polynomial necessary to match the precision of these data.

T, 8C 0 8 16 24 32 40

o, mg/L 14.62 11.84 9.87 8.42 7.31 6.41

cha9792x_ch19_526-562.indd Page 561 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 561 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

562 FOURIER APPROXIMATION

19.16 Use the Excel Data Analysis Toolpack to fi t a straight line to the following data. Determine the 90% confi dence interval for the intercept. If it encompasses zero, redo the regression, but with the intercept forced to be zero (this is an option on the Regression dialogue box).

x 2 4 6 8 10 12 14

y 6.5 7 13 17.8 19 25.8 26.9

19.17 (a) Use MATLAB to fi t a cubic spline to the following data:

x 0 2 4 7 10 12

y 20 20 12 7 6 6

Determine the value of y at x 5 1.5. (b) Repeat (a), but with zero fi rst derivatives at the end knots. Note that the MATLAB help facility describes how to prescribe end derivatives. 19.18 Use MATLAB to generate 64 points from the function

f (t) 5 cos(10t) 1 sin(3t)

from t 5 0 to 2p. Add a random component to the signal with the function randn. Take an FFT of these values and plot the results. 19.19 In a fashion similar to Sec. 19.8.2, use MATLAB to fi t the data from Prob. 19.15 using (a) linear interpolation, (b) a third- order regression polynomial, and (c) a spline. Use each approach to predict oxygen concentration at T 5 10. 19.20 Runge’s function is written as

f (x) 5 1

1 1 25x2

Generate 9 equidistantly spaced values of this function over the interval: [21, 1]. Fit these data with (a) an eighth-order polyno- mial, (b) a linear spline, and (c) a cubic spline. Present your results graphically. 19.21 A dye is injected into the circulating blood volume to mea- sure a patient’s cardiac output, which is the volume fl ow rate of blood out of the left ventricle of the heart. In other words, cardiac output is the number of liters of blood your heart pumps in a min- ute. For a person at rest, the rate might be 5 or 6 liters per minute. If you are a trained marathon runner running a marathon, your cardiac output can be as high as 30 L/min. The data below shows the response of an individual when 5 mg of dye was injected into the venous system.

Time (s) 2 6 9 12 15 18 20 24

Concentration (mg/L) 0 1.5 3.2 4.1 3.4 2 1 0

Fit a polynomial curve through the data points and use the function to approximate the patient’s cardiac output, which can be calculated by:

Cardiac output 5 amount of dye

area under curve a L

min b

FIGURE P19.22

f (t)

t 0.250 0.5

–1

0

1

0.75 1

19.22 In electric circuits, it is common to see current behavior in the form of a square ware as shown in Fig. P19.22. Solving for the Fourier series from

f(t) 5 e A0 0 # t # Ty2 2A0 Ty2 # t # T

we get the Fourier series

f (t) 5 a q

n51 a 4A0

(2n 2 1)p b sin a2p(2n 2 1)t

T b

Let A0 5 1 and T 5 0.25 s. Plot the fi rst six terms of the Fourier series individually, as well as the sum of these six terms. Use a package such as Excel or MATLAB if possible. 19.23 Develop a plot of the following data with (a) sixth-order interpolating polynomial, (b) a cubic spline, and (c) a cubic spline with zero end derivatives.

x 0 100 200 400 600 800 1000

f(x) 0 0.82436 1.00000 0.73576 0.40601 0.19915 0.09158

In each case, compare your plot with the following equation, which was used to generate these data

f (x) 5 x

200 e2

x 200

11

cha9792x_ch19_526-562.indd Page 562 30/10/13 8:25 PM f-512 cha9792x_ch19_526-562.indd Page 562 30/10/13 8:25 PM f-512 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20 C H A P T E R 20

563

Case Studies: Curve Fitting

The purpose of this chapter is to use the numerical methods for curve fi tting to solve some engineering problems. The fi rst application, which is taken from chemical engineer- ing, demonstrates how a nonlinear model can be linearized and fi t to data using linear regression. The second application employs splines to study a problem that has relevance to the environmental area of civil engineering: heat and mass transport in a stratifi ed lake. The third application illustrates how a fast Fourier transform can be employed in electrical engineering to analyze a signal by determining its major harmonics. The fi nal application demonstrates how multiple linear regression is used to analyze experimental data for a fl uids problem taken from mechanical and aerospace engineering.

20.1 LINEAR REGRESSION AND POPULATION MODELS (CHEMICAL/BIO ENGINEERING)

Background. Population growth models are important in many fi elds of engineering. Fundamental to many of the models is the assumption that the rate of change of the popu- lation (dpydt) is proportional to the actual population (p) at any time (t), or in equation form,

dp

dt 5 kp (20.1)

where k 5 a proportionality factor called the specifi c growth rate and has units of time21. If k is a constant, then the solution of Eq. (20.1) can be obtained from the theory of differential equations:

p(t) 5 p0e kt (20.2)

where p0 5 the population when t 5 0. It is observed that p(t) in Eq. (20.2) approaches infi nity as t becomes large. This behavior is clearly impossible for real systems. There- fore, the model must be modifi ed to make it more realistic.

Solution. First, it must be recognized that the specifi c growth rate k cannot be constant as the population becomes large. This is the case because, as p approaches infi nity, the organism being modeled will become limited by factors such as food shortages and toxic waste production. One way to express this mathematically is to use a saturation-growth-rate

cha9792x_ch20_563-581.indd Page 563 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 563 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

564 CASE STUDIES: CURVE FITTING

model such that

k 5 kmax f

K 1 f (20.3)

where kmax 5 the maximum attainable growth rate for large values of food ( f ) and K 5 the half-saturation constant. The plot of Eq. (20.3) in Fig. 20.1 shows that when f 5 K, k 5 kmaxy2. Therefore, K is that amount of available food that supports a population growth rate equal to one-half the maximum rate. The constants K and kmax are empirical values based on experimental measurements of k for various values of f. As an example, suppose the population p represents a yeast employed in the commercial production of beer and f is the concentration of the carbon source to be fermented. Measurements of k versus f for the yeast are shown in Table 20.1.

FIGURE 20.1 Plot of specifi c growth rate versus available food for the saturation-growth-rate model used to characterize microbial kinetics. The value K is called a half-saturation constant because it conforms to the concentration where the specifi c growth rate is half its maximum value.

K

Food available, f

kmax

S p

ec if

ic g

ro w

th r

at e,

k

kmax 2

TABLE 20.1 Data used to evaluate the constants for a saturation-growth-rate model to characterize microbial kinetics.

f, mg/L k, day2l 1/f, L/mg 1/k, day

7 0.29 0.14286 3.448 9 0.37 0.11111 2.703 15 0.48 0.06666 2.083 25 0.65 0.04000 1.538 40 0.80 0.02500 1.250 75 0.97 0.01333 1.031 100 0.99 0.01000 1.010 150 1.07 0.00666 0.935

cha9792x_ch20_563-581.indd Page 564 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 564 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20.1 LINEAR REGRESSION AND POPULATION MODELS 565

It is required to calculate kmax and K from these empirical data. This is accomplished by inverting Eq. (20.3) in a manner similar to Eq. (17.17) to yield

1

k 5

K 1 f

kmax f 5

K

kmax 1

f 1

1

kmax (20.4)

By this manipulation, we have transformed Eq. (20.3) into a linear form, that is, 1yk is a linear function of 1yf, with slope Kykmax and intercept 1ykmax. These values are plotted in Fig. 20.2. Because of this transformation, the linear least-squares procedures described in Chap. 17 can be used to determine kmax 5 1.23 day

2l and K 5 22.18 mg/L. These results combined with Eq. (20.3) are compared to the untransformed data in Fig. 20.3, and when substituted into the model in Eq. (20.1), give

dp

dt 5 1.23

f

22.18 1 f p (20.5)

Note that the fi t yields a sum of the squares of the residuals (as computed for these untransformed data) of 0.001305. Equation (20.5) can be solved using the theory of differential equations or using numerical methods discussed in Chap. 25 when f(t) is known. If f approaches zero as p becomes large, then dpydt approaches zero and the population stabilizes. The linearization of Eq. (20.3) is one way to evaluate the constants kmax and K. An alternative approach, which fi ts the relationship in its original form, is the nonlinear regression described in Sec. 17.5. Figure 20.4 shows how the Excel Solver tool can be used to estimate the parameters with nonlinear regression. As can be seen, a column of predicted values is developed based on the model and the parameter guesses. These are

FIGURE 20.2 Linearized version of the saturation-growth-rate model. The line is a least-squares fi t that is used to evaluate the model coeffi cients kmax 5 1.23 day21 and K 5 22.18 mg/L for a yeast that is used to produce beer.

1/ k,

d ay

0 0.04 0.08 1/f, L/mg

0.12 0.16

1

2

1 kmax

Intercept =

K kmax

Slope =

3

cha9792x_ch20_563-581.indd Page 565 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 565 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

566 CASE STUDIES: CURVE FITTING

used to generate a column of squared residuals that are summed, and the result is placed in cell D14. The Excel Solver is then invoked to minimize cell D14 by adjusting cells B1:B2. The result, as shown in Fig. 20.4, yields estimates of kmax 5 1.23 and K 5 22.14, with an Sr 5 0.001302. Thus, although, as expected, the nonlinear regression yields a slightly better fi t, the results are almost identical. In other applications, this may not be

FIGURE 20.3 Fit of the saturation-growth-rate model to a yeast employed in the commercial production of beer.

k, d

ay –

1

0 50 f, mg/L

100 150

kmax

1

FIGURE 20.4 Nonlinear regression to fi t the saturation-growth-rate model to a yeast employed in the commercial production of beer.

cha9792x_ch20_563-581.indd Page 566 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 566 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20.2 USE OF SPLINES TO ESTIMATE HEAT TRANSFER 567

true (or the function may not be compatible with linearization) and nonlinear regression could represent the only feasible option for obtaining a least-squares fi t.

20.2 USE OF SPLINES TO ESTIMATE HEAT TRANSFER (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. Lakes in the temperate zone can become thermally stratifi ed during the summer. As depicted in Fig. 20.5, warm, buoyant water near the surface overlies colder, denser bottom water. Such stratifi cation effectively divides the lake vertically into two layers: the epilimnion and the hypolimnion separated by a plane called the thermocline. Thermal stratifi cation has great signifi cance for environmental engineers studying the pollution of such systems. In particular, the thermocline greatly diminishes mixing between the two layers. As a result, decomposition of organic matter can lead to severe depletion of oxygen in the isolated bottom waters. The location of the thermocline can be defi ned as the infl ection point of the temperature- depth curve—that is, the point at which d2Tydx2 5 0. It is also the point at which the absolute value of the fi rst derivative or gradient is a maximum. Use cubic splines to deter- mine the thermocline depth for Platte Lake (Table 20.2). Also use the splines to determine the value of the gradient at the thermocline.

Solution. These data are analyzed with a program that was developed based on the pseudocode from Fig. 18.18. The results are displayed in Table 20.3, which lists the

FIGURE 20.5 Temperature versus depth during summer for Platte Lake, Michigan.

z (m

)

0

10

20

30

0 10

Epilimnion

Thermocline

Hypolimnion

� (�C) 20 30

TABLE 20.2 Temperature versus depth during summer for Platte Lake, Michigan.

T, 8C 22.8 22.8 22.8 20.6 13.9 11.7 11.1 11.1

z, m 0 2.3 4.9 9.1 13.7 18.3 22.9 27.2

cha9792x_ch20_563-581.indd Page 567 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 567 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

568 CASE STUDIES: CURVE FITTING

spline predictions along with fi rst and second derivatives at intervals of 1 m down through the water column. The results are plotted in Fig. 20.6. Notice how the thermocline is clearly located at the depth where the gradient is highest (that is, the absolute value of the derivative is greatest) and the second derivative is zero. The depth is 11.35 m and the gradient at this point is 21.618C/m.

TABLE 20.3 Output of spline program based on pseudocode from Fig. 18.18.

Depth (m) T(C) dT/dz d2T/dz2 Depth (m) T(C) dT/dz d2T/dz2 0. 22.8000 2.0115 .0000 15. 12.7652 2.6518 .3004 1. 22.7907 2.0050 .0130 16. 12.2483 2.3973 .2086 2. 22.7944 .0146 .0261 17. 11.9400 2.2346 .1167 3. 22.8203 .0305 2.0085 18. 11.7484 2.1638 .0248 4. 22.8374 2.0055 2.0635 19. 11.5876 2.1599 .0045 5. 22.7909 2.0966 2.1199 20. 11.4316 2.1502 .0148 6. 22.6229 2.2508 2.1884 21. 11.2905 2.1303 .0251 7. 22.2665 2.4735 2.2569 22. 11.1745 2.1001 .0354 8. 21.6531 2.7646 2.3254 23. 11.0938 2.0596 .0436 9. 20.7144 21.1242 2.3939 24. 11.0543 2.0212 .0332 10. 19.4118 21.4524 2.2402 25. 11.0480 .0069 .0229 11. 17.8691 21.6034 2.0618 26. 11.0646 .0245 .0125 12. 16.2646 21.5759 .1166 27. 11.0936 .0318 .0021 13. 14.7766 21.3702 .2950 28. 11.1000 .0000 .0000 14. 13.5825 2.9981 .3923

FIGURE 20.6 Plots of (a) temperature, (b) gra- dient, and (c) second derivative versus depth (m) generated with the cubic spline program. The thermocline is located at the infl ection point of the temperature-depth curve.

z, m

0

8

16

4

12

24

20

28

0

(a)

10

Thermocline

T, �C 20 – 2.0

(b)

– 1.0 dT/dz

0.0 – 0.5

(c)

0.0 d2T/dz2

0.5

cha9792x_ch20_563-581.indd Page 568 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 568 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20.3 FOURIER ANALYSIS 569

20.3 FOURIER ANALYSIS (ELECTRICAL ENGINEERING)

Background. Fourier analysis is used in many areas of engineering. However, it is extensively employed in electrical engineering applications such as signal processing. In 1848, Johann Rudolph Wolf devised a method for quantifying solar activity by counting the number of individual spots and groups of spots on the sun’s surface. He computed a quantity, now called a Wolf sunspot number, by adding 10 times the number of groups plus the total count of individual spots. As in Fig. 20.7, the record of this number extends back to 1700. On the basis of the early historical records, Wolf deter- mined the cycle’s length to be 11.1 years. Use a Fourier analysis to confi rm this result by applying an FFT to these data from Fig. 20.7. Pinpoint the period by developing a power versus period plot.

Solution. These data for year and sunspot number were downloaded from the Web1 and stored in a tab-delimited fi le: sunspot.dat. The fi le can be loaded into MATLAB software and the year and number information assigned to vectors of the same name,

>> load sunspot.dat >> year=sunspot(:,1);number=sunspot(:,2);

Next, an FFT can be applied to the sunspot numbers

>> y=fft(number);

After getting rid of the fi rst harmonic, the length of the FFT is determined (n) and then the power and frequency are calculated,

>> y(1)=[ ]; >> n=length(y); >> power=abs(y(1:n/2)).^2; >> nyquist=1/2; >> freq=(1:n/2)/(n/2)*nyquist;

1At the time of this book’s printing, the html was http://www.ngdc.noaa.gov//stp/SOLAR/SSN/ssn.html.

FIGURE 20.7 Plot of Wolf sunspot number versus year.

1700 1800 1900 2000

200

100

0

cha9792x_ch20_563-581.indd Page 569 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 569 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

570 CASE STUDIES: CURVE FITTING

At this point, the power spectrum is a plot of power versus frequency. However, because period is more meaningful in the present context, we can determine the period and a power-period plot,

>> period=1./freq; >> plot(period,power)

The result, as shown in Fig. 20.8, indicates a peak at about 11 years. The exact value can be computed with

>> index=find(power==max(power)); >> period(index)

ans= 10.9259

20.4 ANALYSIS OF EXPERIMENTAL DATA (MECHANICAL/AEROSPACE ENGINEERING)

Background. Engineering design variables are often dependent on several independent variables. Often this functional dependence is best characterized by multivariate power equations. As discussed in Sec. 17.3, a multiple linear regression of log-transformed data provides a means to evaluate such relationships. For example, a mechanical engineering study indicates that fl uid fl ow through a pipe is related to pipe diameter and slope (Table 20.4). Use multiple linear regression to analyze these data. Then use the resulting model to predict the fl ow for a pipe with a diameter of 2.5 ft and a slope of 0.025 ft/ft.

Solution. The power equation to be evaluated is

Q 5 a0 D a1Sa2 (20.6)

FIGURE 20.8 Power spectrum for Wolf sunspot numbers.

0 10 Period (years)

20 30

2

1

P o

w er

( �

10 7 )

0

cha9792x_ch20_563-581.indd Page 570 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 570 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

20.4 ANALYSIS OF EXPERIMENTAL DATA 571

where Q 5 fl ow (ft3/s), S 5 slope (ft/ft), D 5 pipe diameter (ft), and a0, a1, and a2 5 coef- fi cients. Taking the logarithm of this equation yields

log Q 5 log a0 1 a1 log D 1 a2 log S

In this form, the equation is suited for multiple linear regression because log Q is a linear function of log S and log D. Using the logarithm (base 10) of the data in Table 20.4, we can generate the following normal equations expressed in matrix form [recall Eq. (17.22)]:

£ 9 2.334 218.903

2.334 0.954 24.903

218.903 24.903 44.079

§ • log a0

a1 a2

¶ 5 • 11.691

3.945

222.207

This system can be solved using Gauss elimination for

log a0 5 1.7475

a1 5 2.62

a2 5 0.54

Since log a0 5 1.7475, then a0 5 10 1.7475 5 55.9 and Eq. (20.6) is

Q 5 55.9D2.62S 0.54 (20.7)

Eq. (20.7) can be used to predict fl ow for the case of D 5 2.5 ft and S 5 0.025 ft/ft, as in

Q 5 55.9(2.5)2.62(0.025)S0.54 5 84.1 ft3/s

It should be noted that Eq. (20.7) can be used for other purposes besides computing fl ow. For example, the slope is related to head loss hL and pipe length L by S 5 hLyL. If this relationship is substituted into Eq. (20.7) and the resulting formula solved for hL, the following equation can be developed:

hL 5 L

1721 Q1.85D4.85

This relationship is called the Hazen-Williams equation.

TABLE 20.4 Experimental data for diameter, slope, and fl ow of concrete circular pipes.

Experiment Diameter, ft Slope, ft/ft Flow, ft3/s

1 1 0.001 1.4 2 2 0.001 8.3 3 3 0.001 24.2 4 1 0.01 4.7 5 2 0.01 28.9 6 3 0.01 84.0 7 1 0.05 11.1 8 2 0.05 69.0 9 3 0.05 200.0

cha9792x_ch20_563-581.indd Page 571 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 571 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

572 CASE STUDIES: CURVE FITTING

PROBLEMS

Chemical/Bio Engineering 20.1 Perform the same computation as in Sec. 20.1, but use linear regression and transformations to fi t the data with a power equa- tion. Assess the result. 20.2 You perform experiments and determine the following values of heat capacity c at various temperatures T for a gas:

T 250 230 0 60 90 110

c 1270 1280 1350 1480 1580 1700

Use regression to determine a model to predict c as a function of T. 20.3 It is known that the tensile strength of a plastic increases as a function of the time it is heat-treated. The following data are collected:

Time 10 15 20 25 40 50 55 60 75

Tensile strength 5 20 18 40 33 54 70 60 78

(a) Fit a straight line to these data and use the equation to deter- mine the tensile strength at a time of 32 min.

(b) Repeat the analysis but for a straight line with a zero intercept. 20.4 The following data were gathered to determine the relation- ship between pressure and temperature of a fi xed volume of 1 kg of nitrogen. The volume is 10 m3.

T, °C 240 0 40 80 120 160

p, N/m2 6900 8100 9300 10,500 11,700 12,900

Employ the ideal gas law pV 5 nRT to determine R on the basis of these data. Note that for the law, T must be expressed in kelvins. 20.5 The specifi c volume of a superheated steam is listed in steam ta- bles for various temperatures. For example, at a pressure of 3000 lb/in2, absolute:

T, 8F 700 720 740 760 780

v, ft3/lbm 0.0977 0.12184 0.14060 0.15509 0.16643

Determine y at T 5 7508F. 20.6 A reactor is thermally stratifi ed as in the following table:

Depth, m 0 0.5 1.0 1.5 2.0 2.5 3.0

Temperature, 8C 70 68 55 22 13 11 10

As depicted in Fig. P20.6, the tank can be idealized as two zones separated by a strong temperature gradient or thermocline. The depth of this gradient can be defi ned as the infl ection point of the temperature-depth curve—that is, the point at which d2Tydz2 5 0.

At this depth, the heat fl ux from the surface to the bottom layer can be computed with Fourier’s law,

J 5 2k dT

dz

Use a cubic spline fi t of these data to determine the thermocline depth. If k 5 0.02 cal/(s ? cm ? 8C) compute the fl ux across this interface. 20.7 In Alzheimer’s disease, the number of neurons in the cortex de- creases as the disease progresses. The following data were taken to determine the number of neurotransmitter receptors left in a diseased brain. Free neurotransmitter ([F]) was incubated with tissue and the concentration that bound specifi cally to a receptor ([B]) was mea- sured. When binding is specifi c to a receptor, the concentration bound is related to the free concentration by the following relationship:

[B] 5 Bmax[F]

K 1 [F]

Using the data below, determine the parameters that minimize the sum of the squares of the residuals. Also, compute r2.

[F], nM 0.1 0.5 1 5 10 20 50

[B], nM 10.57 36.61 52.93 82.65 89.46 94.35 101.00

20.8 The following data were taken from a stirred tank reactor for the reaction A S B. Use these data to determine the best possible estimates for k01 and E1 for the following kinetic model,

2 dA

dt 5 k01e2

E1

RT A

where R is the gas constant and equals 0.00198 Kcal/mol/K

1

2

3

0 0 20

D ep

th z

, m

Temperature T, �C

Thermocline

40 60

FIGURE P20.6

cha9792x_ch20_563-581.indd Page 572 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 572 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 573

20.12 The molecular weight of a polymer can be determined from its viscosity by the following relationship:

[h] 5 KM ay

where [h] is the intrinsic viscosity of the polymer My is the viscos- ity averaged molecular weight, and K and a are constants specifi c for the polymer. The intrinsic viscosity is determined experimen- tally be determining the effl ux time, or the time it takes for the polymer solution to fl ow between two etched lines in a capillary viscometer, at several different concentrations of dilute polymer, and extrapolating to infi nite dilution. A plot of

t

t0 2 1

c versus c

should yield a straight line, with a y intercept equal to [h]. The concentration of the polymer solution is c, t is the effl ux time of the polymer solution, and t0 is the effl ux time of the solvent without polymer. Using the data below of effl ux times for dilute solutions of polystyrene in methyl ethyl ketone at 258C and the constants K 5 3.9 3 10–4 and a 5 0.58, fi nd the molecular weight of the poly- styrene sample.

Polymer Concentration, g/dL Effl ux Time, s

0 (pure solvent) 83 0.04 89 0.06 95 0.08 104 0.10 114 0.12 126 0.14 139 0.16 155 0.20 191

20.13 On average, the surface area A of human beings is related to weight W and height H. Measurements on a number of individuals give values of A in the following table:

H (cm) 182 180 179 187 189 194 195 193 200

W (kg) 74 88 94 78 84 98 76 86 96

A (m2) 1.92 2.11 2.15 2.02 2.09 2.31 2.02 2.16 2.31

Develop an equation to predict area as a function of height and weight. Use it to estimate the surface area for a 187-cm, 78-kg person.

2dA/dt (moles/L/s) 460 960 2485 1600 1245

A (moles/L) 200 150 50 20 10

T (K) 280 320 450 500 550

20.9 Use the following set of pressure-volume data to fi nd the best possible virial constants (A1 and A2) for the equation of state shown below. R 5 82.05 ml atm/gmol K and T 5 303 K.

PV

RT 5 1 1

A1 V

1 A2 V 2

P (atm) 0.985 1.108 1.363 1.631

V (ml) 25,000 22,200 18,000 15,000

20.10 Concentration data were taken at 15 time points for the polymerization reaction

x A 1 y B S Ax By

We assume the reaction occurs via a complex mechanism consist- ing of many steps. Several models have been hypothesized and the sum of the squares of the residuals had been calculated for the fi ts of the models of these data. The results are shown below. Which model best describe these data (statistically)? Explain your choice.

Model A Model B Model C

Sr 135 105 100 Number of model parameters fi t 2 3 5

20.11 Below are data taken from a batch reactor of bacterial growth (after lag phase was over). The bacteria are allowed to grow as fast as possible for the fi rst 2.5 hours, and then they are induced to produce a recombinant protein, the production of which slows the bacterial growth signifi cantly. The theoretical growth of bacte- ria can be described by

dX

dt 5 mX

where X is the number of bacteria and m is the specifi c growth rate of the bacteria during exponential growth. Based on these data, estimate the specifi c growth rate of the bacteria during the fi rst 2 hours of growth. During the next 4 hours of growth.

Time, hr 0 1 2 3 4 5 6

[Cells], g/L 0.100 0.332 1.102 1.644 2.453 3.660 5.460

cha9792x_ch20_563-581.indd Page 573 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 573 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

574 CASE STUDIES: CURVE FITTING

20.16 Soft tissue follows an exponential deformation behavior in uniaxial tension while it is in the physiologic or normal range of elongation. This can be expressed as

s 5 E0 a

(eae 2 1)

where s 5 stress, e 5 strain, and Eo and a are material constants that are determined experimentally. To evaluate the two material con- stants, the above equation is differentiated with respect to e. Using the above equation establishes the fundamental relationship for soft tissue

ds

de 5 E0 1 as

To evaluate Eo and a, stress-strain data are plotted as dsyde versus s and the intercept and slope of this plot are the two material con- stants, respectively.

In the following table is stress-strain data for heart chordae ten- dineae (small tendons used to hold heart valves closed during con- traction of the heart muscle; these data are from loading the tissue, while different curves are produced on unloading).

20.14 Determine an equation to predict metabolism rate as a func- tion of mass based on the following data:

Animal Mass, kg Metabolism, watts

Cow 400 270 Human 70 82 Sheep 45 50 Hen 2 4.8 Rat 0.3 1.45 Dove 0.16 0.97

20.15 Human blood behaves as a Newtonian fl uid (see Prob. 20.55) in the high shear rate region where g# . 100. In the low shear rate region where g# , 50, the red cells tend to aggregate into what are called rouleaux, which make the fl uid behavior depart from Newtonian. This low shear rate region is called the Casson region, and there is a transition region between the two distinct fl ow regions. In the Casson region as shear rate approaches zero, the shear stress goes to a fi nite value, similar to a Bingham plastic,

g # , 1/s 0.91 3.3 4.1 6.3 9.6 23 36 49 65 105 126 215 315 402

t, N/m2 0.059 0.15 0.19 0.27 0.39 0.87 1.33 1.65 2.11 3.44 4.12 7.02 10.21 13.01

Region Casson Transition Newtonian

s, 103 N/m2 87.8 96.6 176 263 351 571 834 1229 1624 2107 2678 3380 4258

e, 1023 m/m 153 204 255 306 357 408 459 510 561 612 663 714 765

which is called the yield stress, ty, and this stress must be overcome in order to initiate fl ow in stagnate blood. Flow in the Casson region is usually plotted as the square root of shear rate versus the square root of shear stress, and follows a straight line relationship when plotted in this way. The Casson relationship is

1t 5 1ty 1 Kc1g #

where Kc 5 consistency index. In the table below are experimen- tally measured values of g# and t from a single blood sample over the Casson and Newtonian fl ow regions.

Calculate the derivative dsyde using fi nite differences. Plot these data and eliminate the data points near the zero points that appear not to follow the straight-line relationship. The error in these data comes from the inability of the instrumentation to read the small values in this region. Perform a regression analysis of the remain- ing data points to determine the values of Eo and a.

Plot the stress versus strain data points along with the analytic curve expressed by the fi rst equation. This will indicate how well the analytic curve matches these data.

Many times this does not work well because the value of Eo is diffi cult to evaluate using this technique. To solve this problem Eo

Find the values of Kc and ty using linear regression in the Casson region, and fi nd m by using linear regression in the Newtonian re- gion. Also fi nd the correlation coeffi cient for each regression analy- sis. Plot the two regression lines on a Casson plot (1g# versus 1t) and extend the regression lines as dashed lines into adjoining re- gions; also include the data points in the plot. Limit the shear rate region to 0 , 1g# , 15.

is not used. A data point is selected (s, e) that is in the middle of the regression analysis range. These values are substituted into the fi rst equation and a value for Eoya is determined and substituted into the fi rst equation, which becomes

s 5 a s eae 2 1

b (eae 2 1)

cha9792x_ch20_563-581.indd Page 574 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 574 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 575

Civil/Environmental Engineering 20.19 The shear stresses, in kilopascals (kPa), of nine specimens taken at various depths in a clay stratum are listed below. Estimate the shear stress at a depth of 4.5 m.

Depth, m 1.9 3.1 4.2 5.1 5.8 6.9 8.1 9.3 10.0

Stress, kPa 14.4 28.7 19.2 43.1 33.5 52.7 71.8 62.2 76.6

20.20 A transportation engineering study was conducted to deter- mine the proper design of bike lanes. Data were gathered on bike- lane widths and average distance between bikes and passing cars. The data from nine streets are

Distance, m 2.4 1.5 2.4 1.8 1.8 2.9 1.2 3 1.2

Lane width, m 2.9 2.1 2.3 2.1 1.8 2.7 1.5 2.9 1.5

(a) Plot these data. (b) Fit a straight line to these data with linear regression. Add this

line to the plot. (c) If the minimum safe average distance between bikes and pass-

ing cars is considered to be 2 m, determine the corresponding minimum lane width.

20.21 The saturation concentration of dissolved oxygen in water as a function of temperature and chloride concentration is listed in Table P20.21. Use interpolation to estimate the dissolved oxygen level for T 5 188C with chloride 5 10 g/L. 20.22 For the data in Table P20.21, use polynomial regression to derive a third-order predictive equation for dissolved oxygen con- centration as a function of temperature for the case where chloride concentration is equal to 10 g/L. Use the equation to estimate the dissolved oxygen concentration for T 5 88C. 20.23 Use multiple linear regression to derive a predictive equa- tion for dissolved oxygen concentration as a function of tempera- ture and chloride based on the data from Table P20.21. Use the equation to estimate the concentration of dissolved oxygen for a chloride concentration of 5 g/L at T 5 178C. 20.24 As compared to the models from Probs. 20.22 and 20.23, a somewhat more sophisticated model that accounts for the effect of both temperature and chloride on dissolved oxygen saturation can be hypothesized as being of the form,

os 5 a0 1 f3(T) 1 f1(c)

That is, a constant plus a third-order polynomial in temperature and a linear relationship in chloride are assumed to yield superior re- sults. Use the general linear least-squares approach to fi t this model to the data in Table P20.21. Use the resulting equation to estimate the dissolved oxygen concentration for a chloride concentration of 10 g/L at T 5 208C.

Using this approach, experimental data that are well defi ned will produce a good match of the data points and the analytic curve. Use this new relationship and again plot the stress versus strain data points and this new analytic curve. 20.17 The thickness of the retina changes during certain eye dis- eases. One way to measure retinal thickness is to shine a low-energy laser at the retina and record the refl ections on fi lm. Because of the optical properties of the eye, the refl ections from the front surface of the retina and the back surface of the retina will appear as two lines on the fi lm separated by a distance. The distance between the lines on the fi lm is proportional to the thickness of the retina. Below are data taken from the scanned fi lm. Fit two Gaussian-shaped curves of arbi- trary height and location to these data and determine the distance between the centers of the two peaks. A Gaussian curve has the form

f (x) 5 ke2k

2(x2a)2

1p

where k and a are constants that relate to the peak height and the center of the peak, respectively.

Light Light Position Intensity Position Intensity

0.17 5.10 0.31 25.31 0.18 5.10 0.32 23.79 0.19 5.20 0.33 18.44 0.20 5.87 0.34 12.45 0.21 8.72 0.35 8.22 0.22 16.04 0.36 6.12 0.23 26.35 0.37 5.35 0.24 31.63 0.38 5.15 0.25 26.51 0.39 5.10 0.26 16.68 0.40 5.10 0.27 10.80 0.41 5.09 0.28 11.26 0.42 5.09 0.29 16.05 0.43 5.09 0.3 21.96 0.44 5.09

20.18 The data tabulated below were generated from an experi- ment initially containing pure ammonium cyanate (NH4OCN). It is known that such concentration changes can be modeled by the fol- lowing equation:

c 5 c0

1 1 kc0t

where c0 and k are parameters. Use a transformation to linearize this equation. Then use linear regression to predict the concentra- tion at t 5 160 min.

t (min) 0 20 50 65 150

c (mole/L) 0.381 0.264 0.180 0.151 0.086

cha9792x_ch20_563-581.indd Page 575 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 575 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

576 CASE STUDIES: CURVE FITTING

The concentration of chlorophyll a indicates how much plant life is suspended in the water. As such, it indicates how unclear and unsightly the water appears. Use the above data to determine the relationship of c as a function of p. Use this equation to predict the level of chlorophyll that can be expected if waste treatment is used to lower the phosphorus concentration of western Lake Erie to 10 mg/m3. 20.27 The vertical stress sz under the corner of a rectangular area subjected to a uniform load of intensity q is given by the solution of Boussinesq’s equation:

s 5 q

4p c 2mn2m2 1 n2 1 1

m2 1 n2 1 1 1 m2n2 m2 1 n2 1 2

m2 1 n2 1 1

1 sin21 a 2mn2m2 1 n2 1 1 m2 1 n2 1 1 1 m2n2

b d Because this equation is inconvenient to solve manually, it has been reformulated as

sz 5 q fz(m, n)

TABLE P20.21 Dissolved oxygen concentration in water as a function of temperature (8C) and chloride concentration (g/L).

Dissolved Oxygen (mg/L) for Temperature (8C) and Concentration of Chloride (g/L)

T, 8C c 5 0 g/L c 5 10 g/L c 5 20 g/L

0 14.6 12.9 11.4 5 12.8 11.3 10.3 10 11.3 10.1 8.96 15 10.1 9.03 8.08 20 9.09 8.17 7.35 25 8.26 7.46 6.73 30 7.56 6.85 6.20

20.25 In water-resources engineering, the sizing of reservoirs depends on accurate estimates of water fl ow in the river that is being impounded. For some rivers, long-term historical records of such fl ow data are diffi cult to obtain. In contrast, meteorologi- cal data on precipitation is often available for many years past. Therefore, it is often useful to determine a relationship between fl ow and precipitation. This relationship can then be used to esti- mate fl ows for years when only precipitation measurements were made. The following data are available for a river that is to be dammed:

Precipitation, cm 88.9 108.5104.1139.7 127 94 116.8 99.1

Flow, m3/s 14.6 16.7 15.3 23.2 19.5 16.1 18.1 16.6

(a) Plot these data. (b) Fit a straight line to these data with linear regression. Superim-

pose this line on your plot. (c) Use the best-fi t line to predict the annual water fl ow if the pre-

cipitation is 120 cm. (d) If the drainage area is 1100 km2, estimate what fraction of the

precipitation is lost via processes such as evaporation, deep groundwater infi ltration, and consumptive use.

20.26 The concentration of total phosphorus (p in mg/m3) and chlorophyll a (c in mg/m3) for each of the Great Lakes in 1970 was

p c

Lake Superior 4.5 0.8 Lake Michigan 8.0 2.0 Lake Huron 5.5 1.2 Lake Erie: West basin 39.0 11.0 Central basin 19.5 4.4 East basin 17.5 3.8 Lake Ontario 21.0 5.5

b

z a

�z

FIGURE P20.27

cha9792x_ch20_563-581.indd Page 576 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 576 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 577

be substituted into Hooke’s law to determine the mast’s defl ection, DL 5 strain 3 L, where L 5 the mast’s length. If the wind force is 25,000 N, use these data to estimate the defl ection of a 9-m mast. 20.30 Enzymatic reactions are used extensively to characterize biologically mediated reactions in environmental engineering. Pro- posed rate expressions for an enzymatic reaction are given below where [S] is the substrate concentration and v0 is the initial rate of reaction. Which formula best fi ts the experimental data?

y0 5 k[S] y0 5 k[S]

K 1 [S]  y0 5

k[S]2

K 1 [S]2  y0 5

k[S]3

K 1 [S]3

[S], M Initial Rate, 1026 M/s

0.01 6.3636 3 1025

0.05 7.9520 3 1023

0.1 6.3472 3 1022

0.5 6.0049 1 17.690 5 24.425 10 24.491 50 24.500 100 24.500

20.31 Environmental engineers dealing with the impacts of acid rain must determine the value of the ion product of water Kw as a function of temperature. Scientists have suggested the following equation to model this relationship:

2log10 Kw 5 a

Ta 1 b log10 Ta 1 cTa 1 d

where Ta 5 absolute temperature (K), and a, b, c, and d are param- eters. Employ the following data and regression to estimate the parameters:

where fz(m, n) is called the infl uence value and m and n are dimensionless ratios, with m 5 ayz and n 5 byz and a and b as defi ned in Fig. P20.27. The infl uence value is then tabulated, a portion of which is given in Table P20.27. If a 5 4.6 and b 5 14, use a third-order interpolating polynomial to compute sz at a depth 10 m below the corner of a rectangular footing that is subject to a total load of 100 t (metric tons). Express your an- swer in tonnes per square meter. Note that q is equal to the load per area. 20.28 Three disease-carrying organisms decay exponentially in lake water according to the following model:

p(t) 5 Ae21.5t 1 Be20.3t 1 Ce20.05t

Estimate the initial population of each organism (A, B, and C) given the following measurements:

t, hr 0.5 1 2 3 4 5 6 7 9

p(t ) 6.0 4.4 3.2 2.7 2.2 1.9 1.7 1.4 1.1

TABLE P20.27

m n 5 1.2 n 5 1.4 n 5 1.6

0.1 0.02926 0.03007 0.03058 0.2 0.05733 0.05894 0.05994 0.3 0.08323 0.08561 0.08709 0.4 0.10631 0.10941 0.11135 0.5 0.12626 0.13003 0.13241 0.6 0.14309 0.14749 0.15027 0.7 0.15703 0.16199 0.16515 0.8 0.16843 0.17389 0.17739

20.29 The mast of a sailboat has a cross-sectional area of 10.65 cm2 and is constructed of an experimental aluminum alloy. Tests were performed to defi ne the relationship between stress and strain. The test results are

Strain, cm/cm 0.0032 0.0045 0.0055 0.0016 0.0085 0.0005

Stress, N/cm2 4970 5170 5500 3590 6900 1240

The stress caused by wind can be computed as F/Ac; where F 5 force in the mast and Ac 5 mast’s cross-sectional area. This value can then

Ta (K) 273.15 283.15 293.15 303.15 313.15

Kw 1.164 3 10215 2.950 3 10215 6.846 3 10215 1.467 3 10214 2.929 3 10214

20.32 An environmental engineer has reported the data tabulated below for an experiment to determine the growth rate of bacteria k as a function of oxygen concentration c. It is known that such data can be modeled by the following equation

k 5 kmaxc

2

cs 1 c 2

where cs and kmax are parameters. Use a transformation to linearize this equation. Then use linear regression to estimate cs and kmax and predict the growth rate at c 5 2 mg/L.

cha9792x_ch20_563-581.indd Page 577 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 577 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

578 CASE STUDIES: CURVE FITTING

(a) On the basis of a linear regression of these data, determine current for a voltage of 3.5 V. Plot the line and the data and evaluate the fi t.

(b) Redo the regression and force the intercept to be zero. 20.40 It is known that the voltage drop across an inductor follows Faraday’s law:

VL 5 L di

dt

where VL is the voltage drop (in volts), L is inductance (in henrys; 1 H 5 1 V ? s/A), and i is current (in amperes). Employ the follow- ing data to estimate L:

di/dt, A/s 1 2 4 6 8 10

VL, V 5.5 12.5 17.5 32 38 49

What is the meaning, if any, of the intercept of the regression equa- tion derived from these data? 20.41 Ohm’s law states that the voltage drop V across an ideal re- sistor is linearly proportional to the current i fl owing through the resistor as in V 5 iR, where R is the resistance. However, real resis- tors may not always obey Ohm’s law. Suppose that you performed some very precise experiments to measure the voltage drop and corresponding current for a resistor. The results, as listed in Table P20.41, suggest a curvilinear relationship rather than the straight line represented by Ohm’s law. In order to quantify this relation- ship, a curve must be fi t to these data. Because of measurement er- ror, regression would typically be the preferred method of curve fi tting for analyzing such experimental data. However, the smooth- ness of the relationship, as well as the precision of the experimental methods, suggests that interpolation might be appropriate. Use Newton’s interpolating polynomial to fi t these data and compute V for i 5 0.10. What is the order of the polynomial that was used to generate the data?

TABLE P20.41 Experimental data for voltage drop across a resistor subjected to various levels of current.

i 22 21 20.5 0.5 1 2

V 2637 296.5 220.5 20.5 96.5 637

20.42 Repeat Prob. 20.41 but determine the coeffi cients of the polynomial (Sec. 18.4) that fi t the data in Table P20.41. 20.43 An experiment is performed to determine the percent elon- gation of electrical conducting material as a function of tempera- ture. The resulting data are listed below. Predict the percent elongation for a temperature of 4008C.

Temperature, 8C 200 250 300 375 425 475 600

% elongation 7.5 8.6 8.7 10 11.3 12.7 15.3

c (mg/L) 0.5 0.8 1.5 2.5 4

k (per day) 1.1 2.4 5.3 7.6 8.9

20.33 The following model is frequently used in environmental engineering to parameterize the effect of temperature T (8C) on biochemical reaction rates k (per day),

k 5 k20u T220

where k20 and u are parameters. Use a transformation to linearize this equation. Then employ linear regression to estimate k20 and u and predict the reaction rate at T 5 178C.

T (8C) 6 12 18 24 30

k (per day) 0.14 0.20 0.31 0.46 0.69

20.34 As a member of Engineers Without Borders, you are working in a community that has contaminated drinking water. At t 5 0, you add a disinfectant to a cistern that is contaminated with bacteria. You make the following measurements at several times thereafter:

t (hrs) 2 4 6 8 10

c (#/100 mL) 430 190 80 35 16

If the water is safe to drink when the concentration falls below 5 #/100 mL, estimate the time at which the concentration will fall below this limit.

Electrical Engineering 20.35 Perform the same computations as in Sec. 20.3, but analyze data generated with f(t) 5 4 cos(5t) 2 7 sin(3t) 1 6. 20.36 You measure the voltage drop V across a resistor for a num- ber of different values of current i. The results are

i 0.25 0.75 1.25 1.5 2.0

V 20.45 20.6 0.70 1.88 6.0

Use fi rst- through fourth-order polynomial interpolation to estimate the voltage drop for i 5 1.15. Interpret your results. 20.37 Duplicate the computation for Prob. 20.36, but use polyno- mial regression to derive best fi t equations of order 1 through 4 using all the data. Plot and evaluate your results. 20.38 The current in a wire is measured with great precision as a function of time:

t 0 0.1250 0.2500 0.3750 0.5000

i 0 6.24 7.75 4.85 0.0000

Determine i at t 5 0.23. 20.39 The following data was taken from an experiment that mea- sured the current in a wire for various imposed voltages:

V, V 2 3 4 5 7 10

i, A 5.2 7.8 10.7 13 19.3 27.5

cha9792x_ch20_563-581.indd Page 578 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 578 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 579

experimentally by placing known weights onto the spring and measuring the resulting compression. Such data were contained in Table P20.49 and plotted in Fig. P20.49. Notice that above a weight of 40 3 104 N, the linear relationship between the force and displacement breaks down. This sort of behavior is typical of what is termed a “hardening spring.” Employ linear regression to deter- mine a value of k for the linear portion of this system. In addition, fi t a nonlinear relationship to the nonlinear portion. 20.50 Repeat Prob. 20.49 but fi t a power curve to all the data in Table P20.49. Comment on your results. 20.51 The distance required to stop an automobile consists of both thinking and braking components each of which is a function of its speed. The following experimental data was collected to quantify this relationship. Develop a best-fi t equation for both the thinking

20.44 Bessel functions often arise in advanced engineering analy- ses such as the study of electric fi elds. These functions are usually not amenable to straightforward evaluation and, therefore, are often compiled in standard mathematical tables. For example,

x 1.8 2 2.2 2.4 2.6

J1(x) 0.5815 0.5767 0.556 0.5202 0.4708

Estimate J1(2.1), (a) using an interpolating polynomial and (b) using cubic splines. Note that the true value is 0.568292. 20.45 The population (p) of a small community on the outskirts of a city grows rapidly over a 20-year period:

t 0 5 10 15 20

p 100 200 450 950 2000

As an engineer working for a utility company, you must forecast the population 5 years into the future in order to anticipate the de- mand for power. Employ an exponential model and linear regres- sion to make this prediction.

Mechanical/Aerospace Engineering 20.46 Based on Table 20.4, use linear and quadratic interpolation to compute Q for D 5 1.23 ft and S 5 0.001 ft/ft. Compare your results with the same value computed with the formula derived in Sec. 20.4. 20.47 Reproduce Sec. 20.4, but develop an equation to predict slope as a function of diameter and fl ow. Compare your results with the formula from Sec. 20.4 and discuss your results. 20.48 Dynamic viscosity of water m(1023 N ? s/m2) is related to temperature T(8C) in the following manner:

T 0 5 10 20 30 40

m 1.787 1.519 1.307 1.002 0.7975 0.6529

(a) Plot these data. (b) Use interpolation to predict m at T 5 7.58C. (c) Use polynomial regression to fi t a parabola to these data in order

to make the same prediction. 20.49 Hooke’s law, which holds when a spring is not stretched too far, signifi es that the extension of the spring and the applied force are linearly related. The proportionality is parameterized by the spring constant k. A value for this parameter can be established

TABLE P20.49 Experimental values for elongation x and force F for the spring on an automobile suspension system.

Displacement, m 0.10 0.17 0.27 0.35 0.39 0.42 0.43 0.44

Force, 104 N 10 20 30 40 50 60 70 80

Displacement, m

40

0.2

Hooke's law Nonideal behavior: spring is

"hardening"

0.4

Fo rc

e, 1

04 N

FIGURE P20.49 Plot of force (in 104 newtons) versus displacement (in meters) for the spring from the automobile suspension system.

cha9792x_ch20_563-581.indd Page 579 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 579 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

580 CASE STUDIES: CURVE FITTING

A common example is toothpaste. For pseudoplastics, or “shear thinning” fl uids, the shear stress is

raised to a power n less than one,

t 5 mg # n

Such fl uids, such as yogurt, mayonnaise, and shampoo, exhibit a decrease in viscosity with increasing stress. Note that for cases where n . 1, called dilatant (or “shear thickening”) fl uids, viscos- ity actually increases with shear stress. Examples include starch in water and wet beach sand.

The following data show the relationship between the shear stress t and the shear strain rate g

# . The yield stress ty is the amount

of stress that must be exceeded before fl ow begins. Find the viscos- ity m (slope), ty, and the r

2 value using a regression method. What is the type of fl uid?

Stress t, N/m2 3.25 4.25 4.65 5.65 6.05

Shear strain rate g# , 1/s 0.9 2.1 2.9 4.1 4.9

20.56 The relationship between stress t and the shear strain rate g #

for a pseudoplastic fl uid (see Prob. 20.55) can be expressed by the equation t 5 mg

# n. The following data come from a 0.5% hydrox- ethylcellulose in water solution. Using a power-law fi t, fi nd the values of m and n.

Shear strain rate g# , 1/s 50 70 90 110 130

Stress t, N/m2 6.01 7.48 8.59 9.19 10.21

and braking components. Use these equations to estimate the total stopping distance for a car traveling at 110 km/hr.

Speed, km/hr 30 45 60 75 90 120

Thinking, m 5.6 8.5 11.1 14.5 16.7 22.4

Braking, m 5.0 12.3 21.0 32.9 47.6 84.7

20.52 An experiment is performed to defi ne the relationship be- tween applied stress and the time to fracture for a type of stainless steel. Eight different values of stress are applied, and the resulting data are

Applied stress x, kg/mm2 5 10 15 20 25 30 35 40

Fracture time y, hr 40 30 25 40 18 20 22 15

Plot these data and then develop a best-fi t equation to predict the fracture time for an applied stress of 20 kg/mm2. 20.53 The acceleration due to gravity at an altitude y above the surface of the earth is given by

y, m 0 30,000 60,000 90,000 120,000

g, m/s2 9.8100 9.7487 9.6879 9.6278 9.5682

Compute g at y 5 55,000 m. 20.54 The creep rate e# is the time rate at which strain increases, and stress data below were obtained from a testing procedure. Using a power law curve fi t,

e #

5 Bsm

fi nd the value of B and m. Plot your results using a log-log scale.

Creep rate, min–1 0.0004 0.0011 0.0021 0.0031

Stress, MPa 5.775 8.577 10.874 12.555

20.55 It is a common practice when examining a fl uid’s viscous behavior to plot the shear strain rate (velocity gradient)

dy

dy 5 g

#

on the abscissa versus shear stress (t) on the ordinate. When a fl uid has a straight-line behavior between these two variables it is called a Newtonian fl uid, and the resulting relationship is

t 5 mg #

where m is the fl uid viscosity. Many common fl uids follow this be- havior such as water, milk, and oil. Fluids that do not behave in this way are called non-Newtonian. Some examples of non-Newtonian fl uids are shown in Fig. P20.55.

For Bingham plastics, there is a yield stress ty that must be overcome before fl ow will begin,

t 5 ty 1 mg #

FIGURE P20.55

Bin gha

m

Pse udo

plas tic

New ton

ian

Dila tan

t

Shear strain (�• )

S h

ea r

st re

ss (

�)

cha9792x_ch20_563-581.indd Page 580 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 580 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 581

20.59 Develop equations to fi t the ideal specifi c heats cp (kJ/kg ? K), as a function of temperature T (K), for several gases as listed in Table P20.59. 20.60 Temperatures are measured at various points on a heated plate (Table P20.60). Estimate the temperature at (a) x 5 4, y 5 3.2, and (b) x 5 4.3, y 5 2.7. 20.61 The data below were obtained from a creep test performed at room temperature on a wire composed of 40% tin, 60% lead, and solid core solder. This was done by measuring the increase in strain over time while a constant load was applied to a test specimen. Using a linear regression method, fi nd (a) the equation of these line that best fi ts these data and (b) the r2 value. Plot your results. Does the line pass through the origin—that is, at time zero—should there be any strain? If the line does not pass through the origin, force it to do so. Does this new line represent the data trend? Suggest a new equation that satis- fi es zero strain at zero time and also represents the data trend well.

20.57 The velocity u of air fl owing past a fl at surface is measured at several distances y away from the surface. Fit a curve to these data assuming that the velocity is zero at the surface (y 5 0). Use your result to determine the shear stress (m du/dy) at the surface. (m 5 1.8 3 1025 N ? s/m2)

y, m 0.002 0.006 0.012 0.018 0.024

u, m/s 0.287 0.899 1.915 3.048 4.299

20.58 Andrade’s equation has been proposed as a model of the effect of temperature on viscosity,

m 5 DeByTa

where m 5 dynamic viscosity of water (1023 N ? s/m2), Ta 5 absolute temperature (K), and D and B are parameters. Fit this model to the data for water from Prob. 20.48.

TABLE P20.59 Ideal specifi c heats, cp (kJ/kg ? K) as a function of temperature for several gases.

Gas 250 K 300 K 350 K 450 K 550 K 650 K 800 K 900 K 1000 K

H2 14.051 14.307 14.427 14.501 14.53 14.571 14.695 14.822 14.983 CO2 0.791 0.846 0.895 0.978 1.046 1.102 1.169 1.204 1.234 O2 0.913 0.918 0.928 0.956 0.988 1.017 1.054 1.074 1.09 N2 1.039 1.039 1.041 1.049 1.065 1.086 1.121 1.145 1.167

TABLE P20.60 Temperatures (8C) at various points on a square heated plate.

x 5 0 x 5 2 x 5 4 x 5 6 x 5 8

y 5 0 100.00 90.00 80.00 70.00 60.00 y 5 2 85.00 64.49 53.50 48.15 50.00 y 5 4 70.00 48.90 38.43 35.03 40.00 y 5 6 55.00 38.78 30.39 27.07 30.00 y 5 8 40.00 35.00 30.00 25.00 20.00

Time, Strain, Time, Strain, Time, Strain, min % min % min %

0.085 0.10 3.589 0.26 7.092 0.43 0.586 0.13 4.089 0.30 7.592 0.45 1.086 0.16 4.590 0.32 8.093 0.47 1.587 0.18 5.090 0.34 8.593 0.50 2.087 0.20 5.591 0.37 9.094 0.52 2.588 0.23 6.091 0.39 9.594 0.54 3.088 0.25 6.592 0.41 10.097 0.56

cha9792x_ch20_563-581.indd Page 581 28/10/13 8:10 PM f-479 cha9792x_ch20_563-581.indd Page 581 28/10/13 8:10 PM f-479 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

582

EPILOGUE: PART FIVE

PT5.4 TRADE-OFFS

Table PT5.4 provides a summary of the trade-offs involved in curve fi tting. The tech- niques are divided into two broad categories, depending on the uncertainty of the data. For imprecise measurements, regression is used to develop a “best” curve that fi ts the overall trend of the data without necessarily passing through any of the individual points. For precise measurements, interpolation is used to develop a curve that passes directly through each of the points. All the regression methods are designed to fi t functions that minimize the sum of the squares of the residuals between the data and the function. Such methods are termed least-squares regression. Linear least-squares regression is used for cases where a depen- dent and an independent variable are related to each other in a linear fashion. For situ- ations where a dependent and an independent variable exhibit a curvilinear relationship, several options are available. In some cases, transformations can be used to linearize the relationship. In these instances, linear regression can be applied to the transformed vari- ables to determine the best straight line. Alternatively, polynomial regression can be employed to fi t a curve directly to the data. Multiple linear regression is utilized when a dependent variable is a linear function of two or more independent variables. Logarithmic transformations can also be applied to this type of regression for some cases where the multiple dependency is curvilinear.

TABLE PT5.4 Comparison of the characteristics of alternative methods for curve fi tting.

Error Match of Number of Associated Individual Points Matched Programming Method with Data Data Points Exactly Effort Comments

Regression Linear regression Large Approximate 0 Easy Polynomial regression Large Approximate 0 Moderate Round-off error becomes pro-

nounced for higher-order versions Multiple linear regression Large Approximate 0 Moderate Nonlinear regression Large Approximate 0 Diffi cult Interpolation Newton’s Small Exact n 1 1 Easy Usually preferred for exploratory divided-difference analyses polynomials Lagrange polynomials Small Exact n 1 1 Easy Usually preferred when order is known Cubic splines Small Exact Piecewise fi t of Moderate First and second derivatives equal data points at knots

cha9792x_ep05_582-585.indd Page 582 29/10/13 9:52 PM F-468 cha9792x_ep05_582-585.indd Page 582 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.5 IMPORTANT RELATIONSHIPS AND FORMULAS 583

Polynomial and multiple linear regression (note that simple linear regression is a member of both) belong to a more general class of linear least-squares models. They are classifi ed in this way because they are linear with respect to their coeffi cients. These models are typically implemented using linear algebraic systems that are sometimes ill-conditioned. However, in many engineering applications (that is, for lower-order fi ts), this does not come into play. For cases where it is a problem, alternative approaches are available. For example, a technique called orthogonal polynomials is available to perform polynomial regression (see Sec. PT5.6). Equations that are not linear with respect to their coeffi cients are called nonlinear. Special regression techniques are available to fi t such equations. These are approximate methods that start with initial parameter estimates and then iteratively home in on values that minimize the sum of the squares. Polynomial interpolation is designed to fi t a unique nth-order polynomial that passes exactly through n 1 1 precise data points. This polynomial is presented in two alterna- tive formats. Newton’s divided-difference interpolating polynomial is ideally suited for those cases where the proper order of the polynomial is unknown. Newton’s polynomial is appropriate for such situations because it is easily programmed in a format to compare results with different orders. In addition, an error estimate can be simply incorporated into the technique. Thus, you can compare and choose from results using several different- order polynomials. The Lagrange interpolating polynomial is an alternative formulation that is appropri- ate when the order is known a priori. For these situations, the Lagrange version is somewhat simpler to program and does not require the computation and storage of fi nite divided differences. Another approach to curve fi tting is spline interpolation. This technique fi ts a low- order polynomial to each interval between data points. The fi t is made smooth by setting the derivatives of adjacent polynomials to the same value at their connecting points. The cubic spline is the most common version. Splines are of great utility when fi tting data that is generally smooth but exhibits local areas of abrupt change. Such data tends to induce wild oscillations in higher-order interpolating polynomials. Cubic splines are less prone to these oscillations because they are limited to third-order variations. Beyond the one-dimensional case, interpolation can be implemented for multidimen- sional data. Both interpolating polynomials and splines can be used for this purpose. Software packages are available to expedite such applications. The fi nal method covered in this part of the book is Fourier approximation. This area deals with using trigonometric functions to approximate waveforms. In contrast to the other techniques, the major emphasis of this approach is not to fi t a curve to data points. Rather, the curve fi t is employed to analyze the frequency characteristics of a signal. In particular, a fast Fourier transform is available to very effi ciently transform a function from the time to the frequency domain to elucidate its underlying harmonic structure.

PT5.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT5.5 summarizes important information that was presented in Part Five. This table can be consulted to quickly access important relationships and formulas.

cha9792x_ep05_582-585.indd Page 583 29/10/13 9:52 PM F-468 cha9792x_ep05_582-585.indd Page 583 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

584 EPILOGUE: PART FIVE

syyx 5 B Sr

n 2 2

r2 5 St 2 Sr

St

syyx 5 B Sr

n 2 (m 1 1)

r2 5 St 2 Sr

St

syyx 5 B Sr

n 2 (m 1 1)

r2 5 St 2 Sr

St

R2 5 (x 2 x0) (x 2 x1) (x 2 x2) f  (3)(j)

6

or R2 5 (x 2 x0) (x 2 x1) (x 2 x2)f 3x3, x2, x1, x0]

R2 5 (x 2 x0) (x 2 x1) (x 2 x2) f  (3)(j)

6

or

R2 5 (x 2 x0) (x 2 x1) (x 2 x2)f  3x3, x2, x1, x0]

Linear regression

Polynomial regression

Multiple linear regression

Newton’s divided- difference interpolating polynomial*

Lagrange interpolating polynomial*

Cubic splines

y 5 a0 1 a1x

where a1 5 ngxiyi 2 gxigyi ngx2i 2 (gxi)2

a0 5 y 2 a1x

y 5 a0 1 a1x 1 p 1 am x m

(Evaluation of a’s equivalent to solution of m 1 1 linear algebraic equations)

y 5 a0 1 a1x1 1 p 1 am x m (Evaluation of a’s equivalent to solution of m 1 1 linear algebraic equations)

f2(x) 5 b0 1 b1(x 2 x0) 1 b2(x 2 x0)(x 2 x1) where b0 5 f (x0)

b1 5 f [x1, x0] b2 5 f [x2, x1, x0]

f2(x) 5 f (x0)a x 2 x1x0 2 x1b a x 2 x2 x0 2 x2

b

1 f  (x1)a x 2 x0x1 2 x0b a x 2 x2 x1 2 x2

b

1 f (x2)a x 2 x0x2 2 x0b a x 2 x1 x2 2 x1

b A cubic: aix3 1 bix2 1 cix 1 di is fi t to each interval between knots. First and second derivatives are equal at each knot

TABLE PT5.5 Summary of important information presented in Part Five.

Interpretation Method Formulation Graphical Errors

y

x y

x y

x1

x2

y

x

y

x

a1 x 3 + b1 x

2 + c1 x + d1

a2 x 3 + b2 x

2 + c2 x + d2

knot

y

x

*Note: For simplicity, second-order versions are shown.

PT5.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

Although polynomial regression with normal equations is adequate for many engineering applications, there are problem contexts where its sensitivity to round-off error can rep- resent a serious limitation. An alternative approach based on orthogonal polynomials can mitigate these effects. It should be noted that this approach does not yield a best-fi t equation, but rather, yields individual predictions for given values of the independent

cha9792x_ep05_582-585.indd Page 584 29/10/13 9:52 PM F-468 cha9792x_ep05_582-585.indd Page 584 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT5.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 585

variable. Information on orthogonal polynomials can be found in Shampine and Allen (1973) and Guest (1961). Whereas the orthogonal polynomial technique is helpful for developing a polynomial regression, it does not represent a solution to the instability problem for the general linear regression model [Eq. (17.23)]. An alternative approach based on single-value decomposition, called the SVD method, is available for this purpose. Forsythe et al. (1977), Lawson and Hanson (1974), and Press et al. (2007) contain information on this approach. In addition to the Gauss-Newton algorithm, there are a number of optimization methods that can be used to directly develop a least-squares fi t for a nonlinear equation. These nonlinear regression techniques include Marquardt’s and the steepest-descent methods (recall Part Four). General information on regression can be found in Draper and Smith (1981). All the methods in Part Five have been couched in terms of fi tting a curve to data points. In addition, you may also desire to fi t a curve to another curve. The primary motivation for such functional approximation is to represent a complicated function by a simpler version that is easier to manipulate. One way to do this is to use the compli- cated function to generate a table of values. Then the techniques discussed in this part of the book can be used to fi t polynomials to these discrete values. An alternative approach is based on the minimax principle (recall Fig. 17.2c). This principle specifi es that the coeffi cients of the approximating polynomial be chosen so that the maximum discrepancy is as small as possible. Thus, although the approximation may not be as good as that given by the Taylor series at the base point, it is generally better across the entire range of the fi t. Chebyshev economization is an example of an approach for functional approximation based on such a strategy (Ralston and Rabinowitz, 1978; Gerald and Wheatley, 2004; and Carnahan, Luther, and Wilkes, 1969). An important area in curve fi tting is the combining of splines with least-squares regression. Thus, a cubic spline is generated that does not intercept every point, but rather, minimizes the sum of the squares of the residuals between the data points and the spline curves. The approach involves using the so-called B splines as basis functions. These are so named because of their use as basis function but also because of their characteristic bell shape. Such curves are consistent with a spline approach in that their value and their fi rst and second derivatives would have continuity at their extremes. Thus, continuity of f(x) and its lower derivatives at the knots is ensured. Wold (1974), Prenter (1975), and Cheney and Kincaid (2008) present discussions of this approach. In summary, the foregoing is intended to provide you with avenues for deeper ex- ploration of the subject. Additionally, all the above references provide descriptions of the basic techniques covered in Part Five. We urge you to consult these alternative sources to broaden your understanding of numerical methods for curve fi tting.

cha9792x_ep05_582-585.indd Page 585 29/10/13 9:52 PM F-468 cha9792x_ep05_582-585.indd Page 585 29/10/13 9:52 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART SIX

cha9792x_p06_586-602.indd Page 586 06/11/13 5:51 PM F-468 cha9792x_p06_586-602.indd Page 586 06/11/13 5:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

587

PT6.1 MOTIVATION

Calculus is the mathematics of change. Because engineers must continuously deal with sys- tems and processes that change, calculus is an essential tool of our profession. Standing at the heart of calculus are the related mathematical concepts of differentiation and integration. According to the dictionary defi nition, to differentiate means “to mark off by differences; distinguish; . . . to perceive the difference in or between.” Mathematically, the derivative, which serves as the fundamental vehicle for differentiation, represents the rate of change of a dependent variable with respect to an independent variable. As depicted in Fig. PT6.1, the mathematical defi nition of the derivative begins with a difference approximation:

¢y ¢x

5 f(xi 1 ¢x) 2 f(xi)

¢x (PT6.1)

where y and f(x) are alternative representatives for the dependent variable and x is the independent variable. If Dx is allowed to approach zero, as occurs in moving from Fig. PT6.1a to c, the difference becomes a derivative

dy

dx 5 lim ¢xS0

f(xi 1 ¢x) 2 f(xi) ¢x

NUMERICAL DIFFERENTIATION AND INTEGRATION

FIGURE PT6.1 The graphical defi nition of a derivative: as Dx approaches zero in going from (a) to (c), the difference approximation becomes a derivative.

f (xi)

xi xi + �x x

�x

f (xi + �x)

y y y

�y

f (xi) f ' (xi)

f (xi + �x)

xi xi + �x x

�x

(a) (b)

�y

xi x

(c)

cha9792x_p06_586-602.indd Page 587 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 587 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

588 NUMERICAL DIFFERENTIATION AND INTEGRATION

where dyydx [which can also be designated as y9 or f9(xi)] is the fi rst derivative of y with respect to x evaluated at xi. As seen in the visual depiction of Fig. PT6.1c, the derivative is the slope of the tangent to the curve at xi. The second derivative represents the derivative of the fi rst derivative,

d 2y

dx2 5

d

dx ady

dx b

Thus, the second derivative tells us how fast the slope is changing. It is commonly referred to as the curvature, because a high value for the second derivative means high curvature. Finally, partial derivatives are used for functions that depend on more than one variable. Partial derivatives can be thought of as taking the derivative of the function at a point with all but one variable held constant. For example, given a function f that depends on both x and y, the partial derivative of f with respect to x at an arbitrary point (x, y) is defi ned as

0f 0x

5 lim ¢xS0

f(x 1 ¢x, y) 2 f (x, y) ¢x

Similarly, the partial derivative of f with respect to y is defi ned as

0f 0y

5 lim ¢yS0

f(x, y 1 ¢y) 2 f(x, y) ¢y

To get an intuitive grasp of partial derivatives, recognize that a function that depends on two variables is a surface rather than a curve. Suppose you are mountain climbing and have access to a function, f, that yields elevation as a function of longitude (the east-west oriented x-axis) and latitude (the north-south oriented y-axis). If you stop at a particular point, (x0, y0), the slope to the east would be 0f(x0, y0)y0x and the slope to the north would be 0f(x0, y0)y0y. The inverse process to differentiation in calculus is integration. According to the dictionary defi nition, to integrate means “to bring together, as parts, into a whole; to unite; to indicate the total amount . . . .” Mathematically, integration is represented by

I 5 # b

a f(x) dx (PT6.2)

which stands for the integral of the function f(x) with respect to the independent variable x, evaluated between the limits x 5 a to x 5 b. The function f(x) in Eq. (PT6.2) is referred to as the integrand. As suggested by the dictionary defi nition, the “meaning” of Eq. (PT6.2) is the total value, or summation, of f(x) dx over the range x 5 a to b. In fact, the symbol e is actu- ally a stylized capital S that is intended to signify the close connection between integra- tion and summation. Figure PT6.2 represents a graphical manifestation of the concept. For functions lying above the x axis, the integral expressed by Eq. (PT6.2) corresponds to the area under the curve of f(x) between x 5 a and b.1

1It should be noted that the process represented by Eq. (PT6.2) and Fig. PT6.2 is called defi nite integration. There is another type called indefi nite integration in which the limits a and b are unspecifi ed. As will be discussed in Part Seven, indefi nite integration deals with determining a function whose derivative is given.

cha9792x_p06_586-602.indd Page 588 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 588 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.1 MOTIVATION 589

As outlined above, the “marking off” or “discrimination” of differentiation and the “bringing together” of integration are closely linked processes that are, in fact, inversely related (Fig. PT6.3). For example, if we are given a function y(t) that specifi es an object’s position as a function of time, differentiation provides a means to determine its velocity, as in (Fig. PT6.3a).

y(t) 5 d

dt y(t)

Conversely, if we are provided with velocity as a function of time, integration can be used to determine its position (Fig. PT6.3b),

y(t) 5 # t

0 y(t) dt

Thus, we can make the general claim that the evaluation of the integral

I 5 # b

a f(x) dx

is equivalent to solving the differential equation

dy

dx 5 f(x)

for y(b) given the initial condition y(a) 5 0. Because of this close relationship, we have opted to devote this part of the book to both processes. Among other things, this will provide the opportunity to highlight their

f (x)

a b x

FIGURE PT6.2 Graphical representation of the integral of f(x) between the limits x 5 a to b. The integral is equivalent to the area under the curve.

cha9792x_p06_586-602.indd Page 589 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 589 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

590 NUMERICAL DIFFERENTIATION AND INTEGRATION

similarities and differences from a numerical perspective. In addition, our discussion will have relevance to the next parts of the book where we will cover differential equations.

PT6.1.1 Noncomputer Methods for Differentiation and Integration

The function to be differentiated or integrated will typically be in one of the following three forms:

1. A simple continuous function such as a polynomial, an exponential, or a trigonomet- ric function.

2. A complicated continuous function that is diffi cult or impossible to differentiate or integrate directly.

3. A tabulated function, where values of x and f(x) are given at a number of discrete points, as is often the case with experimental or fi eld data.

In the fi rst case, the derivative or integral of a simple function may be evaluated analytically using calculus. For the second case, analytical solutions are often impractical, and sometimes impossible, to obtain. In these instances, as well as in the third case of discrete data, approximate methods must be employed. A noncomputer method for determining derivatives from data is called equal-area graphical differentiation. In this method, the (x, y) data are tabulated and, for each in- terval, a simple divided difference DyyDx is employed to estimate the slope. Then these values are plotted as a stepped curve versus x (Fig. PT6.4). Next, a smooth curve is drawn that attempts to approximate the area under the stepped curve. That is, it is drawn

FIGURE PT6.3 The contrast between (a) differ- entiation and (b) integration.

y

0 0

200

400

8 124 t

v

0 0

2

4

8 124 t

v

0 0

2

4

8 124

(a) t

y

0 0

200

400

8 124

(b) t

cha9792x_p06_586-602.indd Page 590 06/11/13 5:51 PM F-468 cha9792x_p06_586-602.indd Page 590 06/11/13 5:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.1 MOTIVATION 591

so that visually, the positive and negative areas are balanced. The rates at given values of x can then be read from the curve. In the same spirit, visually oriented approaches were employed to integrate tabulated data and complicated functions in the precomputer era. A simple intuitive approach is to plot the function on a grid (Fig. PT6.5) and count the number of boxes that approximate

x

�y �x

x

0

3

6

9

15

18

�y/�x

66.7

50

40

30

23.3

y

0

200

350

470

650

720

dy/dx

76.50

57.50

45.00

36.25

25.00

21.50

x

0

3

6

9

15

18

0 3

50

6 9 (b) (c)(a)

12 15 18

FIGURE PT6.4 Equal-area differentiation. (a) Centered fi nite divided dif- ferences are used to estimate the derivative for each interval between the data points. (b) The derivative estimates are plotted as a bar graph. A smooth curve is superimposed on this plot to approximate the area under the bar graph. This is accomplished by drawing the curve so that equal positive and negative areas are balanced. (c) Values of dyydx can then be read off the smooth curve.

FIGURE PT6.5 The use of a grid to approxi- mate an integral.

f (x)

a b x

cha9792x_p06_586-602.indd Page 591 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 591 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

592 NUMERICAL DIFFERENTIATION AND INTEGRATION

the area. This number multiplied by the area of each box provides a rough estimate of the total area under the curve. This estimate can be refi ned, at the expense of additional effort, by using a fi ner grid. Another commonsense approach is to divide the area into vertical segments, or strips, with a height equal to the function value at the midpoint of each strip (Fig. PT6.6). The area of the rectangles can then be calculated and summed to estimate the total area. In this approach, it is assumed that the value at the midpoint provides a valid approximation of the average height of the function for each strip. As with the grid method, refi ned estimates are possible by using more (and thinner) strips to approximate the integral. Although such simple approaches have utility for quick estimates, alternative nu- merical techniques are available for the same purpose. Not surprisingly, the simplest of these methods is similar in spirit to the noncomputer techniques. For differentiation, the most fundamental numerical techniques use fi nite divided differences to estimate derivatives. For data with error, an alternative approach is to fi t a smooth curve to these data with a technique such as least-squares regression and then differentiate this curve to obtain derivative estimates. In a similar spirit, numerical integration or quadrature methods are available to obtain integrals. These methods, which are actually easier to implement than the grid approach, are similar in spirit to the strip method. That is, function heights are multiplied by strip widths and summed to estimate the integral. However, through clever choices of weighting factors, the resulting estimate can be made more accurate than that from the simple strip method. As in the simple strip method, numerical integration and differentiation techniques utilize data at discrete points. Because tabulated information is already in such a form, it is naturally compatible with many of the numerical approaches. Although continuous functions are not originally in discrete form, it is usually a simple proposition to use the given equation to generate a table of values. As depicted in Fig. PT6.7, this table can then be evaluated with a numerical method.

f (x)

a b x

FIGURE PT6.6 The use of rectangles, or strips, to approximate the integral.

cha9792x_p06_586-602.indd Page 592 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 592 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.1 MOTIVATION 593

PT6.1.2 Numerical Differentiation and Integration in Engineering

The differentiation and integration of a function has so many engineering applications that you were required to take differential and integral calculus in your fi rst year at college. Many specifi c examples of such applications could be given in all fi elds of engineering. Differentiation is commonplace in engineering because so much of our work in- volves characterizing the changes of variables in both time and space. In fact, many of the laws and other generalizations that fi gure so prominently in our work are based on the predictable ways in which change manifests itself in the physical world. A prime example is Newton’s second law, which is not couched in terms of the position of an object but rather in its change of position with respect to time. Aside from such temporal examples, numerous laws governing the spatial behavior of variables are expressed in terms of derivatives. Among the most common of these are those laws involving potentials or gradients. For example, Fourier’s law of heat conduction

x

f (x)

f (x)

2.599 2.414 1.945 1.993

x

0.25 0.75 1.25 1.75

0 0

1

2

1

Discrete points Continuous function

2

(c)

(b)

(a) 2 + cos (1 + x 3/2)

1 + 0.5 sin x e 0.5x dx

0 � 2

FIGURE PT6.7 Application of a numerical inte- gration method: (a) A compli- cated, continuous function. (b) Table of discrete values of f(x) generated from the function. (c) Use of a numerical method (the strip method here) to estimate the integral on the basis of the discrete points. For a tabulated function, the data are already in tabular form (b); therefore, step (a) is unnecessary.

cha9792x_p06_586-602.indd Page 593 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 593 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

594 NUMERICAL DIFFERENTIATION AND INTEGRATION

quantifi es the observation that heat fl ows from regions of high to low temperature. For the one-dimensional case, this can be expressed mathematically as

Heat flux 5 2k¿ dT

dx

Thus, the derivative provides a measure of the intensity of the temperature change, or gradient, that drives the transfer of heat. Similar laws provide workable models in many other areas of engineering, including the modeling of fl uid dynamics, mass transfer, chemical reaction kinetics, and electromagnetic fl ux. The ability to accu- rately estimate derivatives is an important facet of our capability to work effectively in these areas. Just as accurate estimates of derivatives are important in engineering, the calculation of integrals is equally valuable. A number of examples relate directly to the idea of the integral as the area under a curve. Figure PT6.8 depicts a few cases where integration is used for this purpose. Other common applications relate to the analogy between integration and summa- tion. For example, a common application is to determine the mean of continuous func- tions. In Part Five, you were introduced to the concept of the mean of n discrete data points [recall Eq. (PT5.1)]:

Mean 5 a

n

i51 yi

n (PT6.3)

FIGURE PT6.8 Examples of how integration is used to evaluate areas in engineering applications. (a) A surveyor might need to know the area of a fi eld bounded by a meandering stream and two roads. (b) A water-resource engineer might need to know the cross-sectional area of a river. (c) A structural engineer might need to determine the net force due to a nonuniform wind blowing against the side of a skyscraper.

(a () b () c)

cha9792x_p06_586-602.indd Page 594 06/11/13 5:51 PM F-468 cha9792x_p06_586-602.indd Page 594 06/11/13 5:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.1 MOTIVATION 595

where yi are individual measurements. The determination of the mean of discrete points is depicted in Fig. PT6.9a. In contrast, suppose that y is a continuous function of an independent variable x, as depicted in Fig. PT6.9b. For this case, there are an infi nite number of values between a and b. Just as Eq. (PT6.3) can be applied to determine the mean of the discrete readings, you might also be interested in computing the mean or average of the continuous func- tion y 5 f(x) for the interval from a to b. Integration is used for this purpose, as speci- fi ed by the formula

Mean 5 #

b

a f(x) dx

b 2 a (PT6.4)

This formula has hundreds of engineering applications. For example, it is used to calcu- late the center of gravity of irregular objects in mechanical and civil engineering and to determine the root-mean-square current in electrical engineering. Integrals are also employed by engineers to evaluate the total amount or quantity of a given physical variable. The integral may be evaluated over a line, an area, or a volume. For example, the total mass of chemical contained in a reactor is given as the product of the concentration of chemical and the reactor volume, or

Mass 5 concentration 3 volume

y

0 4 62

Mean

3 51

ba

(a) i

y = f (x)

Mean

(b)

x FIGURE PT6.9 An illustration of the mean for (a) discrete and (b) continuous data.

cha9792x_p06_586-602.indd Page 595 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 595 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

596 NUMERICAL DIFFERENTIATION AND INTEGRATION

where concentration has units of mass per volume. However, suppose that concentration varies from location to location within the reactor. In this case, it is necessary to sum the products of local concentrations ci and corresponding elemental volumes DVi:

Mass 5 a n

i51 ci ¢Vi

where n is the number of discrete volumes. For the continuous case, where c(x, y, z) is a known function and x, y, and z are independent variables designating position in Cartesian coordinates, integration can be used for the same purpose:

Mass 5 # # # c(x, y, z) dx dy dz or

Mass 5 # # V

# c(V) dV

which is referred to as a volume integral. Notice the strong analogy between summation and integration. Similar examples could be given in other fi elds of engineering. For example, the total rate of energy transfer across a plane where the fl ux (in calories per square centi- meter per second) is a function of position is given by

Heat transfer 5 # A # flux dA

which is referred to as an areal integral, where A 5 area. Similarly, for the one-dimensional case, the total mass of a variable-density rod with constant cross-sectional area is given by

m 5 A# L

0 r(x) dx

where m 5 total weight (kg), L 5 length of the rod (m), r(x) 5 known density (kg/m3) as a function of length x (m), and A 5 cross-sectional area of the rod (m2). Finally, integrals are used to evaluate differential or rate equations. Suppose the velocity of a particle is a known continuous function of time y (t),

dy

dt 5 y(t)

The total distance y traveled by this particle over a time t is given by (Fig. PT6.3b)

y 5 # t

0 y(t) dt (PT6.5)

These are just a few of the applications of differentiation and integration that you might face regularly in the pursuit of your profession. When the functions to be analyzed are simple, you will normally choose to evaluate them analytically. For example, in the falling

cha9792x_p06_586-602.indd Page 596 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 596 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.2 MATHEMATICAL BACKGROUND 597

parachutist problem, we determined the solution for velocity as a function of time [Eq. (1.10)]. This relationship could be substituted into Eq. (PT6.5), which could then be integrated easily to determine how far the parachutist fell over a time period t. For this case, the integral is simple to evaluate. However, it is diffi cult or impossible when the function is complicated, as is typically the case in more realistic examples. In addi- tion, the underlying function is often unknown and defi ned only by measurement at discrete points. For both these cases, you must have the ability to obtain approximate values for derivatives and integrals using numerical techniques. Several such techniques will be discussed in this part of the book.

PT6.2 MATHEMATICAL BACKGROUND

In high school or during your fi rst year of college, you were introduced to differential and integral calculus. There you learned techniques to obtain analytical or exact deriva- tives and integrals. When we differentiate a function analytically, we generate a second function that can be used to compute the derivative for different values of the independent vari- able. General rules are available for this purpose. For example, in the case of the monomial

y 5 xn

the following simple rule applies (n ? 0):

dy

dx 5 nxn21

which is the expression of the more general rule for

y 5 un

where u 5 a function of x. For this equation, the derivative is computed via

dy

dx 5 nun21

du

dx

Two other useful formulas apply to the products and quotients of functions. For example, if the product of two functions of x(u and y) is represented as y 5 uy, then the derivative can be computed as

dy

dx 5 u

dy

dx 1 y

du

dx

For the division, y 5 uyy, the derivative can be computed as

dy

dx 5

y du

dx 2 u

dy

dx

y2

Other useful formulas are summarized in Table PT6.1.

cha9792x_p06_586-602.indd Page 597 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 597 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

598 NUMERICAL DIFFERENTIATION AND INTEGRATION

TABLE PT6.1 Some commonly used derivatives.

d dx

sin x 5 cos x d dx

cot x 5 2csc2 x

d dx

cos x 5 2sin x d dx

sec x 5 sec x tan x

d dx

tan x 5 sec2 x d dx

csc x 5 2csc x cot x

d dx

ln x 5 1 x

d dx

loga x 5 1

x ln a

d dx

ex 5 ex d dx

ax 5 ax ln a

Similar formulas are available for defi nite integration, which deals with determining an integral between specifi ed limits, as in

I 5 # b

a f(x) dx (PT6.6)

According to the fundamental theorem of integral calculus, Eq. (PT6.6) is evaluated as

# b

a f(x) dx 5 F(x)Zba

where F(x) 5 the integral of f(x)—that is, any function such that F9(x) 5 f(x). The nomenclature on the right-hand side stands for

F(x)Z ba 5 F(b) 2 F(a) (PT6.7)

An example of a defi nite integral is

I 5 # 0.8

0

(0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5) dx (PT6.8)

For this case, the function is a simple polynomial that can be integrated analytically by evaluating each term according to the rule

# b

a xn dx 5

x n11

n 1 1 ` b a (PT6.9)

where n cannot equal 21. Applying this rule to each term in Eq. (PT6.8) yields

I 5 0.2x 1 12.5x2 2 200

3 x3 1 168.75x4 2 180x5 1

400

6 x6 ` 0.8

0 (PT6.10)

which can be evaluated according to Eq. (PT6.7) as I 5 1.6405333. This value is equal to the area under the original polynomial [Eq. (PT6.8)] between x 5 0 and 0.8.

cha9792x_p06_586-602.indd Page 598 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 598 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.3 ORIENTATION 599

The foregoing integration depends on knowledge of the rule expressed by Eq. (PT6.9). Other functions follow different rules. These “rules” are all merely instances of antidif- ferentiation, that is, fi nding F(x) so that F9(x) 5 f(x). Consequently, analytical integration depends on prior knowledge of the answer. Such knowledge is acquired by training and experience. Many of the rules are summarized in handbooks and in tables of integrals. We list some commonly encountered integrals in Table PT6.2. However, many functions of practical importance are too complicated to be contained in such tables. One reason why the techniques in the present part of the book are so valuable is that they provide a means to evaluate relationships such as Eq. (PT6.8) without knowledge of the rules.

PT6.3 ORIENTATION

Before proceeding to the numerical methods for integration, some further orientation might be helpful. The following is intended as an overview of the material discussed in Part Six. In addition, we have formulated some objectives to help focus your efforts when studying the material.

PT6.3.1 Scope and Preview

Figure PT6.10 provides an overview of Part Six. Chapter 21 is devoted to the most common approaches for numerical integration—the Newton-Cotes formulas. These

TABLE PT6.2 Some simple integrals that are used in Part Six. The a and b in this table are constants and should not be confused with the limits of integration discussed in the text.

#u dv 5 uv 2 #v du

#un du 5 u n11

n 1 1 1 C  n ? 21

#abx dx 5 a bx

b ln a 1 C  a . 0, a ? 1

#dxx 5 ln 0 x 0 1 C  x ? 0

#sin (ax 1 b) dx 5 21a cos (ax 1 b) 1 C

#cos (ax 1 b) dx 5 1a sin (ax 1 b) 1 C

# ln 0 x 0 dx 5 x ln 0 x 0 2 x 1 C

#eax dx 5 e ax

a 1 C

#xeax dx 5 e ax

a2 (ax 2 1) 1 C

# dxa 1 bx2 5 1

2ab tan21

2ab a

x 1 C

cha9792x_p06_586-602.indd Page 599 29/10/13 9:54 PM F-468 cha9792x_p06_586-602.indd Page 599 29/10/13 9:54 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

600 NUMERICAL DIFFERENTIATION AND INTEGRATION

relationships are based on replacing a complicated function or tabulated data with a simple polynomial that is easy to integrate. Three of the most widely used Newton-Cotes formulas are discussed in detail: the trapezoidal rule, Simpson’s 1y3 rule, and Simpson’s 3y8 rule. All these formulas are designed for cases where the data to be integrated are

FIGURE PT6.10 Schematic of the organization of material in Part Six: Numerical Integration and Differentiation.

PART 6 Numerical

Integration and Differentiation

CHAPTER 23 Numerical

Differentiation

CHAPTER 24 Engineering Case Studies

EPILOGUE

PT 6.2 Mathematical background

PT 6.6 Advanced methods

PT 6.5 Important formulas

24.4 Mechanical engineering

24.3 Electrical

engineering

24.2 Civil

engineering

24.1 Chemical

engineering 23.5 Software packages

23.1 High-accuracy

formulas

23.4 Uncertain

data

23.2 Richardson

extrapolation

PT 6.4 Trade-offs

PT 6.3 Orientation

PT 6.1 Motivation

21.2 Simpson's

rules

21.3 Unequal

segments

21.1 Trapezoidal

rule

23.3 Unequal-spaced

data

21.4 Open

integration

21.5 Multiple integrals

22.1 Newton-Cotes for equations

22.2 Romberg

integration

22.4 Gauss

quadrature

22.3 Adaptive

quadrature

22.5 Improper integrals

CHAPTER 21 Newton-Cotes

Integration Formulas

CHAPTER 22 Integration

of Equations

cha9792x_p06_586-602.indd Page 600 29/10/13 9:55 PM F-468 cha9792x_p06_586-602.indd Page 600 29/10/13 9:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.3 ORIENTATION 601

evenly spaced. In addition, we also include a discussion of numerical integration of unequally spaced data. This is a very important topic because many real-world applica- tions deal with data that are in this form. All the above material relates to closed integration, where the function values at the ends of the limits of integration are known. At the end of Chap. 21, we present open integration formulas, where the integration limits extend beyond the range of the known data. Although they are not commonly used for defi nite integration, open integration formulas are presented here because they are utilized extensively in the solution of or- dinary differential equations in Part Seven. The formulations covered in Chap. 21 can be employed to analyze both tabulated data and equations. Chapter 22 deals with three techniques that are expressly designed to integrate equations and functions: Romberg integration, adaptive quadrature, and Gauss quadrature. Computer algorithms are provided for these methods. In addition, methods for evaluating improper integrals are discussed. In Chap. 23, we present additional information on numerical differentiation to sup- plement the introductory material from Chap. 4. Topics include high-accuracy fi nite- difference formulas, Richardson’s extrapolation, and the differentiation of unequally spaced data. The effect of errors on both numerical differentiation and integration is discussed. Finally, the chapter is concluded with a description of the application of sev- eral software packages for integration and differentiation. Chapter 24 demonstrates how the methods can be applied for problem solving. As with other parts of the book, applications are drawn from all fi elds of engineering. A review section, or epilogue, is included at the end of Part Six. This review includes a discussion of trade-offs that are relevant to implementation in engineering practice. In addition, important formulas are summarized. Finally, we present a short review of ad- vanced methods and alternative references that will facilitate your further studies of numerical differentiation and integration.

PT6.3.2 Goals and Objectives

Study Objectives. After completing Part Six, you should be able to solve many nu- merical integration and differentiation problems and appreciate their application for en- gineering problem solving. You should strive to master several techniques and assess their reliability. You should understand the trade-offs involved in selecting the “best’’ method (or methods) for any particular problem. In addition to these general objectives, the specifi c concepts listed in Table PT6.3 should be assimilated and mastered.

Computer Objectives. You will be provided with software and simple computer algorithms to implement the techniques discussed in Part Six. All have utility as learn- ing tools. Algorithms are provided for most of the other methods in Part Six. This information will allow you to expand your software library to include techniques beyond the trapezoi- dal rule. For example, you may fi nd it useful from a professional viewpoint to have software to implement numerical integration and differentiation of unequally spaced data. You may also want to develop your own software for Simpson’s rules, Romberg integra- tion, adaptive integration, and Gauss quadrature, which are usually more effi cient and accurate than the trapezoidal rule.

cha9792x_p06_586-602.indd Page 601 29/10/13 9:55 PM F-468 cha9792x_p06_586-602.indd Page 601 29/10/13 9:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

602 NUMERICAL DIFFERENTIATION AND INTEGRATION

Finally, one of your most important goals should be to master several of the general- purpose software packages that are widely available. In particular, you should become adept at using these tools to implement numerical methods for engineering problem solving.

TABLE PT6.3 Specifi c study objectives for Part Six.

1. Understand the derivation of the Newton-Cotes formulas; know how to derive the trapezoidal rule and how to set up the derivation of both of Simpson’s rules; recognize that the trapezoidal and Simpson’s 1y3 and 3y8 rules represent the areas under fi rst-, second-, and third-order polynomials, respectively.

2. Know the formulas and error equations for (a) the trapezoidal rule, (b) the multiple-application trapezoidal rule, (c) Simpson’s 1y3 rule, (d) Simpson’s 3y8 rule, and (e) the multiple-application Simpson’s rule. Be able to choose the “best” among these formulas for any particular problem context.

3. Recognize that Simpson’s 1y3 rule is fourth-order accurate even though it is based on only three points; realize that all the even-segment–odd-point Newton-Cotes formulas have similar enhanced accuracy.

4. Know how to evaluate the integral and derivative of unequally spaced data. 5. Recognize the difference between open and closed integration formulas. 6. Understand how to evaluate multiple integrals numerically. 7. Understand the theoretical basis of Richardson extrapolation and how it is applied in the Romberg

integration algorithm and for numerical differentiation. 8. Understand the fundamental difference between Newton-Cotes and Gauss quadrature formulas. 9. Recognize why both Romberg integration, adaptive quadrature, and Gauss quadrature have utility

when integrating equations (as opposed to tabular or discrete data). 10. Know how open integration formulas are employed to evaluate improper integrals. 11. Understand the application of high-accuracy numerical-differentiation formulas. 12. Know how to differentiate unequally spaced data. 13. Recognize the differing effects of data error on the processes of numerical integration and

differentiation.

cha9792x_p06_586-602.indd Page 602 29/10/13 9:55 PM F-468 cha9792x_p06_586-602.indd Page 602 29/10/13 9:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21 C H A P T E R 21

603

Newton-Cotes Integration Formulas

The Newton-Cotes formulas are the most common numerical integration schemes. They are based on the strategy of replacing a complicated function or tabulated data with an approximating function that is easy to integrate:

I 5 # b

a

f(x) dx ># b

a fn(x) dx (21.1)

where fn(x) 5 a polynomial of the form

fn(x) 5 a0 1 a1x 1 p 1 an21xn21 1 an xn

where n is the order of the polynomial. For example, in Fig. 21.1a, a fi rst-order polyno- mial (a straight line) is used as an approximation. In Fig. 21.1b, a parabola is employed for the same purpose. The integral can also be approximated using a series of polynomials applied piece- wise to the function or data over segments of constant length. For example, in Fig. 21.2,

FIGURE 21.1 The approximation of an inte- gral by the area under (a) a sin- gle straight line and (b) a single parabola.

f (x)

a b (a) (b)

x

f (x)

a b x

cha9792x_ch21_603-632.indd Page 603 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 603 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

604 NEWTON-COTES INTEGRATION FORMULAS

three straight-line segments are used to approximate the integral. Higher-order polynomi- als can be utilized for the same purpose. With this background, we now recognize that the “strip method” in Fig. PT6.6 employed a series of zero-order polynomials (that is, constants) to approximate the integral. Closed and open forms of the Newton-Cotes formulas are available. The closed forms are those where the data points at the beginning and end of the limits of integra- tion are known (Fig. 21.3a). The open forms have integration limits that extend beyond the range of the data (Fig. 21.3b). In this sense, they are akin to extrapolation as discussed in Sec. 18.5. Open Newton-Cotes formulas are not generally used for defi nite integration.

FIGURE 21.2 The approximation of an inte- gral by the area under three straight-line segments.

f (x)

a b x

FIGURE 21.3 The difference between (a) closed and (b) open integra- tion formulas.

f (x)

a b

(a) x

f (x)

a b

(b) x

cha9792x_ch21_603-632.indd Page 604 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 604 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.1 THE TRAPEZOIDAL RULE 605

However, they are utilized for evaluating improper integrals and for the solution of ordinary differential equations. This chapter emphasizes the closed forms. However, material on open Newton-Cotes formulas is briefl y introduced at the end of this chapter.

21.1 THE TRAPEZOIDAL RULE

The trapezoidal rule is the fi rst of the Newton-Cotes closed integration formulas. It cor- responds to the case where the polynomial in Eq. (21.1) is fi rst order:

I 5 # b

a f(x) dx >#

b

a f1(x) dx

Recall from Chap. 18 that a straight line can be represented as [Eq. (18.2)]

f1(x) 5 f(a) 1 f(b) 2 f(a)

b 2 a (x 2 a) (21.2)

The area under this straight line is an estimate of the integral of f(x) between the limits a and b:

I 5 # b

a c f(a) 1 f(b) 2 f(a)

b 2 a (x 2 a) ddx

The result of the integration (see Box 21.1 for details) is

I 5 (b 2 a) f(a) 1 f(b)

2 (21.3)

which is called the trapezoidal rule.

Box 21.1 Derivation of Trapezoidal Rule

Before integration, Eq. (21.2) can be expressed as

f1(x) 5 f (b) 2 f (a)

b 2 a x 1 f (a) 2

a f (b) 2 a f (a)

b 2 a

Grouping the last two terms gives

f1(x) 5 f (b) 2 f (a)

b 2 a x 1

b f (a) 2 a f (a) 2 a f (b) 1 a f (a)

b 2 a

or

f1(x) 5 f (b) 2 f (a)

b 2 a x 1

b f (a) 2 a f (b)

b 2 a

which can be integrated between x 5 a and x 5 b to yield

I 5 f (b) 2 f (a)

b 2 a x2

2 1

b f (a) 2 a f (b)

b 2 a x ` b

a

This result can be evaluated to give

I 5 f (b) 2 f (a)

b 2 a (b2 2 a2)

2 1

b f (a) 2 a f (b)

b 2 a (b 2 a)

Now, since b2 2 a2 5 (b 2 a)(b 1 a),

I 5 [ f (b) 2 f (a) ] b 1 a

2 1 b f (a) 2 a f (b)

Multiplying and collecting terms yields

I 5 (b 2 a) f (a) 1 f (b)

2

which is the formula for the trapezoidal rule.

cha9792x_ch21_603-632.indd Page 605 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 605 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

606 NEWTON-COTES INTEGRATION FORMULAS

Geometrically, the trapezoidal rule is equivalent to approximating the area of the trapezoid under the straight line connecting f(a) and f(b) in Fig. 21.4. Recall from geometry that the formula for computing the area of a trapezoid is the height times the average of the bases (Fig. 21.5a). In our case, the concept is the same but the trapezoid is on its side (Fig. 21.5b). Therefore, the integral estimate can be represented as

I > width 3 average height (21.4)

FIGURE 21.4 Graphical depiction of the trapezoidal rule.

f (x)

f (a)

f (b)

a b x

FIGURE 21.5 (a) The formula for computing the area of a trapezoid: height times the average of the bases. (b) For the trapezoidal rule, the concept is the same but the trapezoid is on its side.

(b)(a)

WidthH ei

g h

t

Base

H ei

g h

t

Base H

ei g

h t

cha9792x_ch21_603-632.indd Page 606 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 606 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.1 THE TRAPEZOIDAL RULE 607

or

I > (b 2 a) 3 average height (21.5)

where, for the trapezoidal rule, the average height is the average of the function values at the end points, or [ f(a) 1 f(b)]y2. All the Newton-Cotes closed formulas can be expressed in the general format of Eq. (21.5). In fact, they differ only with respect to the formulation of the average height.

21.1.1 Error of the Trapezoidal Rule

When we employ the integral under a straight-line segment to approximate the integral under a curve, we obviously can incur an error that may be substantial (Fig. 21.6). An estimate for the local truncation error of a single application of the trapezoidal rule is (Box. 21.2)

Et 5 2 1

12 f –(j)(b 2 a)3 (21.6)

where j lies somewhere in the interval from a to b. Equation (21.6) indicates that if the function being integrated is linear, the trapezoidal rule will be exact. Otherwise, for functions with second- and higher-order derivatives (that is, with curvature), some error can occur.

FIGURE 21.6 Graphical depiction of the use of a single application of the trapezoidal rule to approximate the integral of f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5 from x 5 0 to 0.8.

f (x)

0.80

2.0

x

Error

Integral estimate

cha9792x_ch21_603-632.indd Page 607 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 607 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

608 NEWTON-COTES INTEGRATION FORMULAS

EXAMPLE 21.1 Single Application of the Trapezoidal Rule

Problem Statement. Use Eq. (21.3) to numerically integrate

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

from a 5 0 to b 5 0.8. Recall from Sec. PT6.2 that the exact value of the integral can be determined analytically to be 1.640533.

Solution. The function values

f(0) 5 0.2

f(0.8) 5 0.232

can be substituted into Eq. (21.3) to yield

I > 0.8 0.2 1 0.232

2 5 0.1728

which represents an error of

Et 5 1.640533 2 0.1728 5 1.467733

which corresponds to a percent relative error of et 5 89.5%. The reason for this large error is evident from the graphical depiction in Fig. 21.6. Notice that the area under the straight line neglects a signifi cant portion of the integral lying above the line. In actual situations, we would have no foreknowledge of the true value. Therefore, an approximate error estimate is required. To obtain this estimate, the function’s second

Box 21.2 Derivation and Error Estimate of the Trapezoidal Rule

An alternative derivation of the trapezoidal rule is possible by inte- grating the forward Newton-Gregory interpolating polynomial. Re- call that for the fi rst-order version with error term, the integral would be (Box 18.2)

I 5 # b

a c f (a) 1 ¢f (a)a 1 f –(j)

2 a(a 2 1)h2 d  dx (B21.2.1)

To simplify the analysis, realize that because a 5 (x 2 a)yh,

dx 5 h da

Inasmuch as h 5 b 2 a (for the one-segment trapezoidal rule), the limits of integration a and b correspond to 0 and 1, respectively. Therefore, Eq. (B21.2.1) can be expressed as

I 5 h# 1

0 c f (a) 1 ¢f (a)a 1 f –(j)

2 a(a 2 1)h2 d  da

If it is assumed that, for small h, the term f 0(j) is approximately

constant, this equation can be integrated:

I 5 h ca f (a) 1 a2 2

¢f (a) 1 aa3 6

2 a2

4 b f –(j)h2 d 1

0

and evaluated as

I 5 h c f (a) 1 ¢f (a) 2 d 2 1

12 f –(j)h3

Because Df(a) 5 f(b) 2 f(a), the result can be written as

I 5 h f (a) 1 f (b)

2 2

1

12 f –(j)h3

Trapezoidal rule Truncation error

Thus, the fi rst term is the trapezoidal rule and the second is an approximation for the error.

cha9792x_ch21_603-632.indd Page 608 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 608 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.1 THE TRAPEZOIDAL RULE 609

derivative over the interval can be computed by differentiating the original function twice to give

f –(x) 5 2400 1 4050x 2 10,800x2 1 8000x3

The average value of the second derivative can be computed using Eq. (PT6.4):

f –(x) 5 #

0.8

0

(2400 1 4050x 2 10,800x2 1 8000x3) dx

0.8 2 0 5 260

which can be substituted into Eq. (21.6) to yield

Ea 5 2 1

12 (260)(0.8)3 5 2.56

which is of the same order of magnitude and sign as the true error. A discrepancy does exist, however, because of the fact that for an interval of this size, the average second derivative is not necessarily an accurate approximation of f 0(j). Thus, we denote that the error is approximate by using the notation Ea, rather than exact by using Et.

21.1.2 The Multiple-Application Trapezoidal Rule

One way to improve the accuracy of the trapezoidal rule is to divide the integration interval from a to b into a number of segments and apply the method to each segment (Fig. 21.7). The areas of individual segments can then be added to yield the integral for the entire interval. The resulting equations are called multiple-application, or composite, integration formulas. Figure 21.8 shows the general format and nomenclature we will use to characterize multiple-application integrals. There are n 1 1 equally spaced base points (x0, x1, x2, . . . , xn). Consequently, there are n segments of equal width:

h 5 b 2 a

n (21.7)

If a and b are designated as x0 and xn, respectively, the total integral can be repre- sented as

I 5 # x1

x0 f(x) dx 1 #

x2

x1 f(x) dx 1 p 1 #

xn

xn21 f(x) dx

Substituting the trapezoidal rule for each integral yields

I 5 h f(x0) 1 f(x1)

2 1 h

f(x1) 1 f(x2)

2 1 p 1 h

f(xn21) 1 f(xn)

2 (21.8)

or, grouping terms,

I 5 h

2 c f(x0) 1 2a

n21

i51 f(xi) 1 f(xn) d (21.9)

cha9792x_ch21_603-632.indd Page 609 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 609 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

610 NEWTON-COTES INTEGRATION FORMULAS

FIGURE 21.7 Illustration of the multiple-application trapezoidal rule. (a) Two segments, (b) three segments, (c) four segments, and (d) fi ve segments.

f (x)

x0 x1 x2

(a)

f (x)

x0 x1 x3x2

(b)

f (x)

x0 x1 x4x3x2

(c)

f (x)

x0 x1 x5x4x3x2

(d )

cha9792x_ch21_603-632.indd Page 610 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 610 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.1 THE TRAPEZOIDAL RULE 611

or, using Eq. (21.7) to express Eq. (21.9) in the general form of Eq. (21.5),

I 5 (b 2 a)

f(x0) 1 2a n21

i51 f(xi) 1 f(xn)

2n (21.10)

Width Average height

Because the summation of the coeffi cients of f(x) in the numerator divided by 2n is equal to 1, the average height represents a weighted average of the function values. According to Eq. (21.10), the interior points are given twice the weight of the two end points f(x0) and f(xn). An error for the multiple-application trapezoidal rule can be obtained by summing the individual errors for each segment to give

Et 5 2 (b 2 a)3

12n3 a n

i51 f –(ji) (21.11)

FIGURE 21.8 The general format and nomen- clature for multiple-application integrals.

f (x)

f (x0)

f (x1) f (x2)

f (x3)

f (xn – 3) f (xn – 2)

f (xn – 1)

f (xn)

x0 xx1 x2 x3 xn – 3 xn – 2 xn – 1 xn

x0 = a xn = b

b – a nh =

cha9792x_ch21_603-632.indd Page 611 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 611 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

612 NEWTON-COTES INTEGRATION FORMULAS

where f 0(ji) is the second derivative at a point ji located in segment i. This result can be simplifi ed by estimating the mean or average value of the second derivative for the entire interval as [Eq. (PT6.3)]

f – > a

n

i51 f –(ji)

n (21.12)

Therefore, g f –(ji) > n f – and Eq. (21.11) can be rewritten as

Ea 5 2 (b 2 a)3

12n2 f – (21.13)

Thus, if the number of segments is doubled, the truncation error will be quartered. Note that Eq. (21.13) is an approximate error because of the approximate nature of Eq. (21.12).

EXAMPLE 21.2 Multiple-Application Trapezoidal Rule

Problem Statement. Use the two-segment trapezoidal rule to estimate the integral of

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

from a 5 0 to b 5 0.8. Employ Eq. (21.13) to estimate the error. Recall that the correct value for the integral is 1.640533.

Solution. n 5 2 (h 5 0.4):

f(0) 5 0.2  f(0.4) 5 2.456  f(0.8) 5 0.232

I 5 0.8 0.2 1 2(2.456) 1 0.232

4 5 1.0688

Et 5 1.640533 2 1.0688 5 0.57173  et 5 34.9%

Ea 5 2 0.83

12(2)2 (260) 5 0.64

where 260 is the average second derivative determined previously in Example 21.1.

The results of the previous example, along with three- through ten-segment applica- tions of the trapezoidal rule, are summarized in Table 21.1. Notice how the error decreases as the number of segments increases. However, also notice that the rate of decrease is gradual. This is because the error is inversely related to the square of n [Eq. (21.13)]. Therefore, doubling the number of segments quarters the error. In subsequent sections we develop higher-order formulas that are more accurate and that converge more quickly on the true integral as the segments are increased. However, before investigating these formu- las, we will fi rst discuss computer algorithms to implement the trapezoidal rule.

21.1.3 Computer Algorithms for the Trapezoidal Rule

Two simple algorithms for the trapezoidal rule are listed in Fig. 21.9. The fi rst (Fig. 21.9a) is for the single-segment version. The second (Fig. 21.9b) is for the multiple-segment

cha9792x_ch21_603-632.indd Page 612 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 612 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.1 THE TRAPEZOIDAL RULE 613

version with a constant segment width. Note that both are designed for data that are in tabulated form. A general program should have the capability to evaluate known func- tions or equations as well. We will illustrate how functions are handled in Chap. 22.

EXAMPLE 21.3 Evaluating Integrals with the Computer

Problem Statement. Use software based on Fig. 21.9b to solve a problem related to our friend, the falling parachutist. As you recall from Example 1.1, the velocity of the parachutist is given as the following function of time:

y(t) 5 gm

c (1 2 e2(cym)t) (E21.3.1)

where y 5 velocity (m/s), g 5 the gravitational constant of 9.8 m/s2, m 5 mass of the parachutist equal to 68.1 kg, and c 5 the drag coeffi cient of 12.5 kg/s. The model predicts the velocity of the parachutist as a function of time as described in Example 1.l.

TABLE 21.1 Results for multiple-application trapezoidal rule to estimate the integral of f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5 from x 5 0 to 0.8. The exact value is 1.640533.

n h I Et (%)

2 0.4 1.0688 34.9 3 0.2667 1.3695 16.5 4 0.2 1.4848 9.5 5 0.16 1.5399 6.1 6 0.1333 1.5703 4.3 7 0.1143 1.5887 3.2 8 0.1 1.6008 2.4 9 0.0889 1.6091 1.9 10 0.08 1.6150 1.6

(a) Single-segment (b) Multiple-segment

FUNCTION Trap (h, fO, f1) FUNCTION Trapm (h, n, f) Trap 5 h * (fO 1 f1)y2 sum 5 fO END Trap DOFOR i 5 1, n 2 1 sum 5 sum 1 2 * fi END DO sum 5 sum 1 fn Trapm 5 h * sum y 2 END Trapm

FIGURE 21.9 Algorithms for the (a) single-segment and (b) multiple-segment trapezoidal rule.

cha9792x_ch21_603-632.indd Page 613 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 613 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

614 NEWTON-COTES INTEGRATION FORMULAS

Suppose we would like to know how far the parachutist has fallen after a certain time t. This distance is given by [Eq. (PT6.5)]

d 5 # t

0 y(t) dt

where d is the distance in meters. Substituting Eq. (E21.3.1),

d 5 gm

c # t

0

(1 2 e2(cym)t) dt

Use your software to determine this integral with the multiple-segment trapezoidal rule using different numbers of segments. Note that performing the integration analytically and substituting known parameter values results in an exact value of d 5 289.43515 m.

Solution. For the case where n 5 10 segments, a calculated integral of 288.7491 is obtained. Thus, we have attained the integral to three signifi cant digits of accuracy. Results for other numbers of segments can be readily generated.

Segments Segment Size Estimated d, m Et (%)

10 1.0 288.7491 0.237 20 0.5 289.2636 0.0593 50 0.2 289.4076 9.5 3 1023

100 0.1 289.4282 2.4 3 1023

200 0.05 289.4336 5.4 3 1024

500 0.02 289.4348 1.2 3 1024

1000 0.01 289.4360 23.0 3 1024

2000 0.005 289.4369 25.9 3 1024

5000 0.002 289.4337 5.2 3 1024

10,000 0.001 289.4317 1.2 3 1023

Up to about 500 segments, the multiple-application trapezoidal rule attains excellent accuracy. However, notice how the error changes sign and begins to increase in absolute value beyond the 500-segment case. The 10,000-segment case actually seems to be di- verging from the true value. This is due to the intrusion of round-off error because of the great number of computations for this many segments. Thus, the level of precision is limited, and we would never reach the exact result of 289.4351 obtained analytically. This limitation and ways to overcome it will be discussed in further detail in Chap. 22.

Three major conclusions can be drawn from the Example 21.3:

For individual applications with nicely behaved functions, the multiple-segment trapezoidal rule is just fine for attaining the type of accuracy required in many engineering applications.

If high accuracy is required, the multiple-segment trapezoidal rule demands a great deal of computational effort. Although this effort may be negligible for a single application, it could be very important when (a) numerous integrals are being evaluated or (b) where the function itself is time consuming to evaluate. For such cases, more efficient approaches (like those in the remainder of this chapter and the next) may be necessary.

cha9792x_ch21_603-632.indd Page 614 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 614 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.2 SIMPSON’S RULES 615

Finally, round-off errors can limit our ability to determine integrals. This is due both to the machine precision as well as to the numerous computations involved in simple techniques like the multiple-segment trapezoidal rule.

We now turn to one way in which effi ciency is improved. That is, by using higher- order polynomials to approximate the integral.

21.2 SIMPSON’S RULES

Aside from applying the trapezoidal rule with fi ner segmentation, another way to obtain a more accurate estimate of an integral is to use higher-order polynomials to connect the points. For example, if there is an extra point midway between f(a) and f(b), the three points can be connected with a parabola (Fig. 21.10a). If there are two points equally spaced between f(a) and f(b), the four points can be connected with a third-order poly- nomial (Fig. 21.10b). The formulas that result from taking the integrals under these polynomials are called Simpson’s rules.

21.2.1 Simpson’s 1/3 Rule

Simpson’s 1y3 rule results when a second-order interpolating polynomial is substituted into Eq. (21.1):

I 5 # b

a f(x) dx >#

b

a f2(x) dx

If a and b are designated as x0 and x2 and f2(x) is represented by a second-order Lagrange polynomial [Eq. (18.23)], the integral becomes

I 5 # x2

x0

c (x 2 x1) (x 2 x2) (x0 2 x1) (x0 2 x2)

f(x0) 1 (x 2 x0) (x 2 x2)

(x1 2 x0) (x1 2 x2) f(x1)

1 (x 2 x0) (x 2 x1)

(x2 2 x0) (x2 2 x1) f(x2) ddx

FIGURE 21.10 (a) Graphical depiction of Simpson’s 1/3 rule: It consists of taking the area under a parabola connecting three points. (b) Graphical depiction of Simpson’s 3/8 rule: It consists of taking the area under a cubic equation connecting four points.

f (x)

(a) x

f (x)

(b) x

cha9792x_ch21_603-632.indd Page 615 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 615 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

616 NEWTON-COTES INTEGRATION FORMULAS

After integration and algebraic manipulation, the following formula results:

I > h

3 [ f(x0) 1 4 f(x1) 1 f(x2) ] (21.14)

where, for this case, h 5 (b 2 a)y2. This equation is known as Simpson’s 1y3 rule. It is the second Newton-Cotes closed integration formula. The label “1y3” stems from the fact that h is divided by 3 in Eq. (21.14). An alternative derivation is shown in Box 21.3 where the Newton-Gregory polynomial is integrated to obtain the same formula. Simpson’s 1y3 rule can also be expressed using the format of Eq. (21.5):

I > (b 2 a) f(x0) 1 4 f(x1) 1 f(x2)

6 (21.15)

Width Average height

Box 21.3 Derivation and Error Estimate of Simpson’s 1/3 Rule

As was done in Box 21.2 for the trapezoidal rule, Simpson’s 1y3 rule can be derived by integrating the forward Newton-Gregory interpolating polynomial (Box 18.2):

I 5 # x2

x0 c f (x0) 1 ¢f (x0)a 1 ¢

2 f (x0)

2 a(a 2 1)

1 ¢3 f (x0)

6 a(a 2 1)(a 2 2)

1 f (4)(j)

24 a(a 2 1)(a 2 2)(a 2 3)h4 d dx

Notice that we have written the polynomial up to the fourth-order term rather than the third-order term as would be expected. The reason for this will be apparent shortly. Also notice that the limits of integration are from x0 to x2. Therefore, when the simplifying substitutions are made (recall Box 21.2), the integral is from a 5 0 to 2:

I 5 h# 2

0

c f (x0) 1 ¢f (x0)a 1 ¢ 2 f (x0)

2 a(a 2 1)

1 ¢3 f (x0)

6 a(a 2 1)(a 2 2)

1 f (4)(j)

24 a(a 2 1)(a 2 2)(a 2 3)h4 d da

which can be integrated to yield

I 5 h ca f (x0) 1 a 2

2 ¢f (x0) 1 aa

3

6 2 a2

4 b ¢2 f (x0)

1 aa4 24

2 a3

6 1 a2

6 b ¢3 f (x0)

1 a a5 120

2 a4

16 1

11a3

72 2 a2

8 b f (4)(j)h4 d 2

0

and evaluated for the limits to give

I 5 h c 2 f (x0) 1 2¢ f (x0) 1 ¢ 2 f (x0)

3

1 (0)¢3 f (x0) 2 1

90 f (4)(j)h4 d (B21.3.1)

Notice the signifi cant result that the coeffi cient of the third divided difference is zero. Because Df(x0) 5 f(x1) 2 f(x0) and D

2f(x0) 5 f(x2) 2 2f(x1) 1 f(x0), Eq. (B21.3.1) can be rewritten as

I 5 h

3 [ f (x0) 1 4 f (x1) 1 f (x2) ] 2

1

90 f (4)(j)h5

Simpson’s 1y3 Truncation error

Thus, the fi rst term is Simpson’s 1y3 rule and the second is the truncation error. Because the third divided difference dropped out, we obtain the signifi cant result that the formula is third-order accurate.

cha9792x_ch21_603-632.indd Page 616 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 616 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.2 SIMPSON’S RULES 617

where a 5 x0, b 5 x2, and x1 5 the point midway between a and b, which is given by (b 1 a)y2. Notice that, according to Eq. (21.15), the middle point is weighted by two- thirds and the two end points by one-sixth. It can be shown that a single-segment application of Simpson’s 1y3 rule has a trun- cation error of (Box 21.3)

Et 5 2 1

90 h5f (4)(j)

or, because h 5 (b 2 a)y2,

Et 5 2 (b 2 a)5

2880 f (4)(j) (21.16)

where j lies somewhere in the interval from a to b. Thus, Simpson’s 1Y3 rule is more accurate than the trapezoidal rule. However, comparison with Eq. (21.6) indicates that it is more accurate than expected. Rather than being proportional to the third derivative, the error is proportional to the fourth derivative. This is because, as shown in Box 21.3, the coeffi cient of the third-order term goes to zero during the integration of the interpo- lating polynomial. Consequently, Simpson’s 1Y3 rule is third-order accurate even though it is based on only three points. In other words, it yields exact results for cubic polyno- mials even though it is derived from a parabola!

EXAMPLE 21.4 Single Application of Simpson’s 1/3 Rule

Problem Statement. Use Eq. (21.15) to integrate

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

from a 5 0 to b 5 0.8. Recall that the exact integral is 1.640533.

Solution.

f(0) 5 0.2  f(0.4) 5 2.456  f(0.8) 5 0.232

Therefore, Eq. (21.15) can be used to compute

I > 0.8 0.2 1 4(2.456) 1 0.232

6 5 1.367467

which represents an exact error of

Et 5 1.640533 2 1.367467 5 0.2730667  et 5 16.6%

which is approximately 5 times more accurate than for a single application of the trap- ezoidal rule (Example 21.1). The estimated error is [Eq . (21.16)]

Ea 5 2 (0.8)5

2880 (22400) 5 0.2730667

where 22400 is the average fourth derivative for the interval as obtained using Eq. (PT6.4). As was the case in Example 21.1, the error is approximate (Ea) because the average fourth

cha9792x_ch21_603-632.indd Page 617 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 617 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

618 NEWTON-COTES INTEGRATION FORMULAS

derivative is not an exact estimate of f (4)(j). However, because this case deals with a fi fth- order polynomial, the result matches.

21.2.2 The Multiple-Application Simpson’s 1/3 Rule

Just as with the trapezoidal rule, Simpson’s rule can be improved by dividing the integra- tion interval into a number of segments of equal width (Fig. 21.11):

h 5 b 2 a

n (21.17)

The total integral can be represented as

I 5 # x2

x0   f(x) dx 1 #

x4

x2   f(x) dx 1 p 1 #

xn

xn22   f(x) dx

Substituting Simpson’s 1y3 rule for the individual integral yields

I > 2h f(x0) 1 4 f(x1) 1 f(x2)

6 1 2h

f(x2) 1 4 f(x3) 1 f(x4)

6

1 p 1 2h f(xn22) 1 4 f(xn21) 1 f(xn)

6

or, combining terms and using Eq. (21.17),

I > (b 2 a) 

f(x0) 1 4 a n21

i51, 3, 5 f(xi) 1 2 a

n22

j52, 4, 6 f(xj) 1 f(xn)

3n (21.18)

Width Average height

FIGURE 21.11 Graphical representation of the multiple application of Simpson’s 1y3 rule. Note that the method can be employed only if the number of segments is even.

f (x)

xba

cha9792x_ch21_603-632.indd Page 618 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 618 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.2 SIMPSON’S RULES 619

Notice that, as illustrated in Fig. 21.11, an even number of segments must be utilized to implement the method. In addition, the coeffi cients “4” and “2” in Eq. (21.18) might seem peculiar at fi rst glance. However, they follow naturally from Simpson’s 1y3 rule. The odd points represent the middle term for each application and hence carry the weight of 4 from Eq. (21.15). The even points are common to adjacent applications and hence are counted twice. An error estimate for the multiple-application Simpson’s rule is obtained in the same fashion as for the trapezoidal rule by summing the individual errors for the segments and averaging the derivative to yield

Ea 5 2 (b 2 a)5

180n4 f (4) (21.19)

where f  (4) is the average fourth derivative for the interval.

EXAMPLE 21.5 Multiple-Application Version of Simpson’s 1y3 Rule

Problem Statement. Use Eq. (21.18) with n 5 4 to estimate the integral of

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

from a 5 0 to b 5 0.8. Recall that the exact integral is 1.640533.

Solution. n 5 4 (h 5 0.2):

f(0) 5 0.2 f(0.2) 5 1.288

f(0.4) 5 2.456 f(0.6) 5 3.464

f(0.8) 5 0.232

From Eq. (21.18),

I 5 0.8 0.2 1 4(1.288 1 3.464) 1 2(2.456) 1 0.232

12 5 1.623467

Et 5 1.640533 2 1.623467 5 0.017067  et 5 1.04%

The estimated error [Eq. (21.19)] is

Ea 5 2 (0.8)5

180(4)4 (22400) 5 0.017067

The previous example illustrates that the multiple-application version of Simp- son’s 1y3 rule yields very accurate results. For this reason, it is considered superior to the trapezoidal rule for most applications. However, as mentioned previously, it is limited to cases where the values are equispaced. Further, it is limited to situations where there are an even number of segments and an odd number of points. Conse- quently, as discussed in the next section, an odd-segment–even-point formula known

cha9792x_ch21_603-632.indd Page 619 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 619 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

620 NEWTON-COTES INTEGRATION FORMULAS

as Simpson’s 3y8 rule is used in conjunction with the 1y3 rule to permit evaluation of both even and odd numbers of segments.

21.2.3 Simpson’s 3/8 Rule

In a similar manner to the derivation of the trapezoidal and Simpson’s 1y3 rule, a third- order Lagrange polynomial can be fi t to four points and integrated:

I 5 # b

a   f(x) dx > #

b

a   f3(x) dx

to yield

I > 3h

8 [ f(x0) 1 3 f(x1) 1 3 f(x2) 1 f(x3) ]

where h 5 (b 2 a)y3. This equation is called Simpson’s 3y8 rule because h is multiplied by 3y8. It is the third Newton-Cotes closed integration formula. The 3y8 rule can also be expressed in the form of Eq. (21.5):

I > (b 2 a)  f(x0) 1 3 f(x1) 1 3 f(x2) 1 f(x3)

8 (21.20)

Thus, the two interior points are given weights of three-eighths, whereas the end points are weighted with one-eighth. Simpson’s 3y8 rule has an error of

Et 5 2 3

80 h5 f  (4)(j)

or, because h 5 (b 2 a)y3,

Et 5 2 (b 2 a)5

6480 f  (4)(j) (21.21)

Because the denominator of Eq. (21.21) is larger than for Eq. (21.16), the 3y8 rule is somewhat more accurate than the 1y3 rule. Simpson’s 1y3 rule is usually the method of preference because it attains third- order accuracy with three points rather than the four points required for the 3y8 version. However, the 3y8 rule has utility when the number of segments is odd. For instance, in Example 21.5 we used Simpson’s rule to integrate the function for four segments. Suppose that you desired an estimate for five segments. One option would be to use a multiple-application version of the trapezoidal rule as was done in Ex- amples 21.2 and 21.3. This may not be advisable, however, because of the large truncation error associated with this method. An alternative would be to apply Simp- son’s 1y3 rule to the first two segments and Simpson’s 3y8 rule to the last three (Fig. 21.12). In this way, we could obtain an estimate with third-order accuracy across the entire interval.

Width Average height

cha9792x_ch21_603-632.indd Page 620 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 620 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.2 SIMPSON’S RULES 621

EXAMPLE 21.6 Simpson’s 3y8 Rule

Problem Statement.

(a) Use Simpson’s 3y8 rule to integrate

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

from a 5 0 to b 5 0.8. (b) Use it in conjunction with Simpson’s 1y3 rule to integrate the same function for fi ve

segments.

Solution.

(a) A single application of Simpson’s 3y8 rule requires four equally spaced points:

f(0) 5 0.2 f(0.2667) 5 1.432724

f(0.5333) 5 3.487177 f(0.8) 5 0.232

Using Eq. (21.20),

I > 0.8 0.2 1 3(1.432724 1 3.487177) 1 0.232

8 5 1.519170

Et 5 1.640533 2 1.519170 5 0.1213630  et 5 7.4%

Ea 5 2 (0.8)5

6480 (22400) 5 0.1213630

FIGURE 21.12 Illustration of how Simpson’s 1y3 and 3y8 rules can be applied in tandem to handle multiple applications with odd numbers of intervals.

f (x)

x0.80.60.40.2

3/8 rule1/3 rule

0

3

2

1

0

cha9792x_ch21_603-632.indd Page 621 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 621 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

622 NEWTON-COTES INTEGRATION FORMULAS

(b) The data needed for a fi ve-segment application (h 5 0.16) is

f(0) 5 0.2 f(0.16) 5 1.296919

f(0.32) 5 1.743393 f(0.48) 5 3.186015

f(0.64) 5 3.181929 f(0.80) 5 0.232

The integral for the fi rst two segments is obtained using Simpson’s 1y3 rule:

I > 0.32 0.2 1 4(1.296919) 1 1.743393

6 5 0.3803237

For the last three segments, the 3y8 rule can be used to obtain

I > 0.48 1.743393 1 3(3.186015 1 3.181929) 1 0.232

8 5 1.264754

The total integral is computed by summing the two results:

I 5 0.3803237 1 1.264753 5 1.645077

Et 5 1.640533 2 1.645077 5 20.00454383  et 5 20.28%

21.2.4 Computer Algorithms for Simpson’s Rules

Pseudocodes for a number of forms of Simpson’s rule are outlined in Fig. 21.13. Note that all are designed for data that are in tabulated form. A general program should have the capability to evaluate known functions or equations as well. We will illustrate how functions are handled in Chap. 22. Notice that the program in Fig. 21.13d is set up so that either an even or odd num- ber of segments may be used. For the even case, Simpson’s 1y3 rule is applied to each pair of segments, and the results are summed to compute the total integral. For the odd case, Simpson’s 3y8 rule is applied to the last three segments, and the 1y3 rule is applied to all the previous segments.

21.2.5 Higher-Order Newton-Cotes Closed Formulas

As noted previously, the trapezoidal rule and both of Simpson’s rules are members of a family of integrating equations known as the Newton-Cotes closed integration formulas. Some of the formulas are summarized in Table 21.2 along with their truncation-error estimates. Notice that, as was the case with Simpson’s 1y3 and 3y8 rules, the fi ve- and six- point formulas have the same order error. This general characteristic holds for the higher- point formulas and leads to the result that the even-segment–odd-point formulas (for example, 1y3 rule and Boole’s rule) are usually the methods of preference. However, it must also be stressed that, in engineering practice, the higher-order (that is, greater than four-point) formulas are rarely used. Simpson’s rules are suffi cient for most applications. Accuracy can be improved by using the multiple-application version. Furthermore, when the function is known and high accuracy is required, methods such

cha9792x_ch21_603-632.indd Page 622 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 622 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.2 SIMPSON’S RULES 623

(a) FUNCTION Simp13 (h, f0, f1, f2) Simp13 5 2*h* (f014*f11f2) / 6 END Simp13

(b) FUNCTION Simp38 (h, f0, f1, f2, f3) Simp38 5 3*h* (f013*(f11f2)1f3) / 8 END Simp38

(c) FUNCTION Simp13m (h, n, f) sum 5 f(0) DOFOR i 5 1, n 2 2, 2 sum 5 sum 1 4 * fi 1 2 * fi11 END DO sum 5 sum 1 4 * fn21 1 fn Simp13m 5 h * sum / 3 END Simp13m

(d ) FUNCTION SimpInt(a,b,n,f) h 5 (b 2 a) / n IF n 5 1 THEN sum 5 Trap(h,fn21,fn) ELSE m 5 n odd 5 n / 2 2 INT(n / 2) IF odd . 0 AND n . 1 THEN sum 5 sum1Simp38(h,fn23,fn22,fn21,fn) m 5 n 2 3 END IF IF m . 1 THEN sum 5 sum 1 Simp13m(h,m,f) END IF END IF SimpInt 5 sum END SimpInt

TABLE 21.2 Newton-Cotes closed integration formulas. The formulas are presented in the format of Eq. (21.5) so that the weighting of the data points to estimate the average height is apparent. The step size is given by h 5 (b 2 a)/n.

Segments (n) Points Name Formula Truncation Error

1 2 Trapezoidal rule (b 2 a) f (x0) 1 f (x1)

2 2 (1y12)h3f''(j)

2 3 Simpson’s 1/3 rule (b 2 a) f (x0) 1 4f (x1) 1 f (x2)

6 2 (1y90)h5f (4)(j)

3 4 Simpson’s 3/8 rule (b 2 a) f (x0) 1 3f (x1) 1 3f (x2) 1 f (x3)

8 2 (3y80)h5f (4)(j)

4 5 Boole’s rule (b 2 a) 7f (x0) 1 32f (x1) 1 12f (x2) 1 32f (x3) 1 7f (x4)

90 2 (8y945)h7f (6)(j)

5 6 (b 2 a) 19f (x0) 1 75f (x1) 1 50f (x2) 1 50f (x3) 1 75f (x4) 1 19f (x5)

288 2 (275y12,096)h7f (6)(j)

FIGURE 21.13 Pseudocode for Simpson’s rules. (a) Single-application Simpson’s 1y3 rule, (b) single- application Simpson’s 3y8 rule, (c) multiple-application Simpson’s 1y3 rule, and (d) multiple- application Simpson’s rule for both odd and even number of segments. Note that for all cases, n must be $1.

cha9792x_ch21_603-632.indd Page 623 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 623 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

624 NEWTON-COTES INTEGRATION FORMULAS

as Romberg integration or Gauss quadrature, described in Chap. 22, offer viable and attractive alternatives.

21.3 INTEGRATION WITH UNEQUAL SEGMENTS

To this point, all formulas for numerical integration have been based on equally spaced data points. In practice, there are many situations where this assumption does not hold and we must deal with unequal-sized segments. For example, experimentally derived data are often of this type. For these cases, one method is to apply the trapezoidal rule to each segment and sum the results:

I 5 h1 f(x0) 1 f(x1)

2 1 h2

f(x1) 1 f(x2)

2 1 p 1 hn

f(xn21) 1 f(xn)

2 (21.22)

where hi 5 the width of segment i. Note that this was the same approach used for the multiple-application trapezoidal rule. The only difference between Eqs. (21.8) and (21.22) is that the h’s in the former are constant. Consequently, Eq. (21.8) could be simplifi ed by grouping terms to yield Eq. (21.9). Although this simplifi cation cannot be applied to Eq. (21.22), a computer program can be easily developed to accommodate unequal-sized segments. Before describing such an algorithm, we will illustrate in the following ex- ample how Eq. (21.22) is applied to evaluate an integral.

EXAMPLE 21.7 Trapezoidal Rule with Unequal Segments

Problem Statement. The information in Table 21.3 was generated using the same poly- nomial employed in Example 21.1. Use Eq. (21.22) to determine the integral for these data. Recall that the correct answer is 1.640533.

Solution. Applying Eq. (21.22) to these data in Table 21.3 yields

I 5 0.12 1.309729 1 0.2

2 1 0.10

1.305241 1 1.309729

2 1 p 1 0.10

0.232 1 2.363

2

5 0.090584 1 0.130749 1 p 1 0.12975 5 1.594801

which represents an absolute percent relative error of et 5 2.8%.

TABLE 21.3 Data for f (x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5, with unequally spaced values of x.

x f (x) x f(x)

0.0 0.200000 0.44 2.842985 0.12 1.309729 0.54 3.507297 0.22 1.305241 0.64 3.181929 0.32 1.743393 0.70 2.363000 0.36 2.074903 0.80 0.232000 0.40 2.456000

cha9792x_ch21_603-632.indd Page 624 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 624 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.3 INTEGRATION WITH UNEQUAL SEGMENTS 625

The data from Example 21.7 are depicted in Fig. 21.14. Notice that some adjacent segments are of equal width and, consequently, could have been evaluated using Simpson’s rules. This usually leads to more accurate results, as illustrated by the following example.

EXAMPLE 21.8 Inclusion of Simpson’s Rules in the Evaluation of Uneven Data

Problem Statement. Recompute the integral for the data in Table 21.3, but use Simpson’s rules for those segments where they are appropriate.

Solution. The fi rst segment is evaluated with the trapezoidal rule:

I 5 0.12 1.309729 1 0.2

2 5 0.09058376

Because the next two segments from x 5 0.12 to 0.32 are of equal length, their integral can be computed with Simpson’s 1y3 rule:

I 5 0.2 1.743393 1 4(1.305241) 1 1.309729

6 5 0.2758029

The next three segments are also equal and, as such, may be evaluated with the 3y8 rule to give I 5 0.2726863. Similarly, the 1y3 rule can be applied to the two segments from x 5 0.44 to 0.64 to yield I 5 0.6684701. Finally, the last two segments, which are of unequal length, can be evaluated with the trapezoidal rule to give values of 0.1663479 and 0.1297500, respectively. The area of these individual segments can be summed to

FIGURE 21.14 Use of the trapezoidal rule to determine the integral of unevenly spaced data. Notice how the shaded segments could be evaluated with Simpson’s rule to attain higher accuracy.

f (x)

x0.5

1/3 rule

3/8 rule

1/3 rule

0

3

0

cha9792x_ch21_603-632.indd Page 625 29/10/13 9:45 PM F-468 cha9792x_ch21_603-632.indd Page 625 29/10/13 9:45 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

626 NEWTON-COTES INTEGRATION FORMULAS

yield a total integral of 1.603641. This represents an error of et 5 2.2%, which is supe- rior to the result using the trapezoidal rule in Example 21.7.

Computer Program for Unequally Spaced Data. It is a fairly simple proposition to program Eq. (21.22). Such an algorithm is listed in Fig. 21.15a. However, as demonstrated in Example 21.8, the approach is enhanced if it imple- ments Simpson’s rules wherever possible. For this reason, we have developed a second algorithm that incorporates this capability. As depicted in Fig 21.15b, the algorithm checks the length of adjacent segments. If two consecutive segments are of equal length, Simpson’s 1y3 rule is applied. If three are equal, the 3y8 rule is used. When adjacent segments are of unequal length, the trapezoidal rule is implemented.

(a) FUNCTION Trapun (x, y, n) LOCAL i, sum sum 5 0 DOFOR i 5 1, n sum 5 sum 1 (xi 2 xi21)*(yi21 1 yi) /2 END DO Trapun 5 sum END Trapun

(b) FUNCTION Uneven (n,x,f) h 5 x1 2 x0 k 5 1 sum 5 0. DOFOR j 5 1, n hf 5 xj11 2 xj IF ABS (h 2 hf) , .000001 THEN IF k 5 3 THEN sum 5 sum 1 Simp13 (h,fj23,fj22,fj21) k 5 k 2 1 ELSE k 5 k 1 1 END IF ELSE IF k 5 1 THEN sum 5 sum 1 Trap (h,fj21,fj) ELSE IF k 5 2 THEN sum 5 sum 1 Simp13 (h,fj22,fj21,fj) ELSE sum 5 sum 1 Simp38 (h,fj23,fj22,fj21,fj) END IF k 5 1 END IF END IF h 5 hf END DO Uneven 5 sum END Uneven

FIGURE 21.15 Pseudocode for integrating unequally spaced data. (a) Trapezoidal rule and (b) combination Simpson’s and trapezoidal rules.

cha9792x_ch21_603-632.indd Page 626 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 626 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

21.5 MULTIPLE INTEGRALS 627

Thus, not only does it allow evaluation of unequal segment data, but if equally spaced information is used, it reduces to using Simpson’s rules. As such, it represents a basic, all-purpose algorithm for the determination of the integral of tabulated data.

21.4 OPEN INTEGRATION FORMULAS

Recall from Fig 21.3b that open integration formulas have limits that extend beyond the range of these data. Table 21.4 summarizes the Newton-Cotes open integration formulas. The formulas are expressed in the form of Eq. (21.5) so that the weighting factors are evident. As with the closed versions, successive pairs of the formulas have the same- order error. The even-segment–odd-point formulas are usually the methods of preference because they require fewer points to attain the same accuracy as the odd-segment–even- point formulas. The open formulas are not often used for defi nite integration. However, as discussed in Chap. 22, they have utility for analyzing improper integrals. In addition, they will have relevance to our discussion of multistep methods for solving ordinary differential equations in Chap. 26.

21.5 MULTIPLE INTEGRALS

Multiple integrals are widely used in engineering. For example, a general equation to compute the average of a two-dimensional function can be written as (recall Eq. PT6.4)

f 5 #

d

c a #

b

a f(x, y)dxb dy

(d 2 c)(b 2 a) (21.23)

The numerator is called a double integral.

TABLE 21.4 Newton-Cotes open integration formulas. The formulas are presented in the format of Eq. (21.5) so that the weighting of the data points to estimate the average height is apparent. The step size is given by h 5 (b 2 a)/n.

Segments (n) Points Name Formula Truncation Error

2 1 Midpoint method (b 2 a) f (x1) (1y3)h3f''(j)

3 2 (b 2 a) f  (x1) 1 f (x2)

2 (3y4)h3f''(j)

4 3 (b 2 a) 2f  (x1) 2 f  (x2) 1 2f (x3)

3 (14y45)h5f (4)(j)

5 4 (b 2 a) 11f (x1) 1 f  (x2) 1 f  (x3) 1 11f  (x4)

24 (95y144)h5f (4)(j)

6 5 (b 2 a) 11f (x1) 2 14f (x2) 1 26f (x3) 2 14f (x4) 1 11f (x5)

20 (41y140)h7f (6)(j)

cha9792x_ch21_603-632.indd Page 627 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 627 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

628 NEWTON-COTES INTEGRATION FORMULAS

The techniques discussed in this chapter (and the following chapter) can be readily employed to evaluate multiple integrals. A simple example would be to take the double integral of a function over a rectangular area (Fig. 21.16). Recall from calculus that such integrals can be computed as iterated integrals

# d

c

a # b

a f(x, y) dxb dy 5 #

b

a

a # d

c

f(x, y)dyb dx (21.24) Thus, the integral in one of the dimensions is evaluated fi rst. The result of this fi rst in- tegration is integrated in the second dimension. Equation (21.24) states that the order of integration is not important. A numerical double integral would be based on the same idea. First, methods like the multiple-segment trapezoidal or Simpson’s rule would be applied in the fi rst dimension with each value of the second dimension held constant. Then the method would be applied to integrate the second dimension. The approach is illustrated in the following example.

EXAMPLE 21.9 Using Double Integral to Determine Average Temperature

Problem Statement. Suppose that the temperature of a rectangular heated plate is de- scribed by the following function:

T (x, y) 5 2xy 1 2x 2 x2 2 2y2 1 72

If the plate is 8-m long (x dimension) and 6-m wide (y dimension), compute the average temperature.

FIGURE 21.16 Double integral as the area under the function surface.

f(x, y)

a

b

x

c

d

y

cha9792x_ch21_603-632.indd Page 628 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 628 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 629

Solution. First, let us merely use two-segment applications of the trapezoidal rule in each dimension. The temperatures at the necessary x and y values are depicted in Fig. 21.17. Note that a simple average of these values is 47.33. The function can also be evaluated analytically to yield a result of 58.66667. To make the same evaluation numerically, the trapezoidal rule is fi rst implemented along the x dimension for each y value. These values are then integrated along the y dimension to give the fi nal result of 2688. Dividing this by the area yields the average temperature as 2688y(6 3 8) 5 56. Now we can apply a single-segment Simpson’s 1y3 rule in the same fashion. This results in an integral of 2816 and an average of 58.66667, which is exact. Why does this occur? Recall that Simpson’s 1y3 rule yielded perfect results for cubic polynomials. Since the highest order term in the function is second order, the same exact result occurs for the present case. For higher-order algebraic functions as well as transcendental functions, it would be necessary to use multi-segment applications to attain accurate integral estimates. In ad- dition, Chap. 22 introduces techniques that are more effi cient than the Newton-Cotes formulas for evaluating integrals of given functions. These often provide a superior means to implement the numerical integrations for multiple integrals.

40

70

64

0

54

72

48

54

24

(8 – 0) 0 + 2(40) + 48

4

(8 – 0) 54 + 2(70) + 54

4

(8 – 0) 72 + 2(64) + 24

4

(6 – 0) = 2688 256 + 2(496) + 448

4

256

448

496

x

y

FIGURE 21.17 Numerical evaluation of a double integral using the two-segment trapezoidal rule.

PROBLEMS

21.1 Evaluate the following integral:

# py2

0

(6 1 3 cos x) dx

(a) analytically; (b) single application of the trapezoidal rule; (c) multiple-application trapezoidal rule, with n 5 2 and 4; (d) single application of Simpson’s 1y3 rule; (e) multiple-application

Simpson’s 1y3 rule, with n 5 4; (f) single application of Simpson’s 3y8 rule; and (g) multiple-application Simpson’s rule, with n 5 5. For each of the numerical estimates (b) through (g), determine the percent relative error based on (a). 21.2 Evaluate the following integral:

# 3

0 (1 2 e22x) dx

cha9792x_ch21_603-632.indd Page 629 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 629 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

630 NEWTON-COTES INTEGRATION FORMULAS

rule; (f) the midpoint method; (g) the 3-segment–2-point open integra- tion formula; and (h) the 4-segment–3-point open integration formula.

# 3

0 (5 1 3 cos x) dx

Compute percent relative errors for the numerical results. 21.9 Suppose that the upward force of air resistance on a falling object is proportional to the square of the velocity. For this case, the velocity can be computed as

y(t) 5 A

gm

cd tanh a

A

gcd m

tb where cd 5 a second-order drag coeffi cient. (a) If g 5 9.81 m/s

2, m 5 68.1 kg, and cd 5 0.25 kg/m, use analytical integration to determine how far the object falls in 10 s. (b) Make the same evaluation, but evaluate the integral with the multiple-segment trapezoidal rule. Use a suffi ciently high n that you get three signifi cant digits of accuracy. 21.10 Evaluate the integral of the following tabular data with (a) the trapezoidal rule and (b) Simpson’s rules:

x 0 0.1 0.2 0.3 0.4 0.5

f (x) 1 8 4 3.5 5 1

21.11 Evaluate the integral of the following tabular data with (a) the trapezoidal rule and (b) Simpson’s rules:

x 22 0 2 4 6 8 10

f (x) 35 5 210 2 5 3 20

21.12 Determine the mean value of the function

f (x) 5 246 1 45x 2 14x2 1 2x3 2 0.075x4

between x 5 2 and 10 by (a) graphing the function and visually estimating the mean value, (b) using Eq. (PT6.4) and the analytical evaluation of the integral, and (c) using Eq. (PT6.4) and a fi ve- segment version of Simpson’s rule to estimate the integral. Calcu- late the relative percent error. 21.13 The function f(x) 5 2e21.5x can be used to generate the fol- lowing table of unequally spaced data:

x 0 0.05 0.15 0.25 0.35 0.475 0.6

f (x) 2 1.8555 1.5970 1.3746 1.1831 0.9808 0.8131

Evaluate the integral from a 5 0 to b 5 0.6 using (a) analytical means, (b) the trapezoidal rule, and (c) a combination of the trap- ezoidal and Simpson’s rules; employ Simpson’s rules wherever possible to obtain the highest accuracy. For (b) and (c), compute the percent relative error (et). 21.14 Evaluate the following double integral:

# 1

21 #

2

0 (x2 2 2y2 1 xy3) dx dy

(a) analytically; (b) single application of the trapezoidal rule; (c) multiple-application trapezoidal rule, with n 5 2 and 4; (d) sin- gle application of Simpson’s 1y3 rule; (e) multiple-application Simpson’s 1y3 rule, with n 5 4; (f) single application of Simpson’s 3y8 rule; and (g) multiple-application Simpson’s rule, with n 5 5. For each of the numerical estimates (b) through (g), determine the percent relative error based on (a). 21.3 Evaluate the following integral:

# 4

22 (1 2 x 2 4x3 1 2x5) dx

(a) analytically; (b) single application of the trapezoidal rule; (c) composite trapezoidal rule, with n 5 2 and 4; (d) single applica- tion of Simpson’s 1y3 rule; (e) Simpson’s 3y8 rule; and (f) Boole’s rule. For each of the numerical estimates (b) through (f) determine the percent relative error based on (a). 21.4 Integrate the following function analytically and using the trapezoidal rule, with n 5 1, 2, 3, and 4:

# 2

1 (x 1 2yx)2 dx

Use the analytical solution to compute true percent relative errors to evaluate the accuracy of the trapezoidal approximations. 21.5 Integrate the following function both analytically and using Simpson’s rules, with n 5 4 and 5. Discuss the results.

# 5

23 (4x 2 3)3 dx

21.6 Integrate the following function both analytically and numer- ically. Use both the trapezoidal and Simpson’s 1y3 rules to numeri- cally integrate the function. For both cases, use the multiple-application version, with n 5 4. Compute percent relative errors for the numerical results.

# 3

0 x2ex dx

21.7 Integrate the following function both analytically and numeri- cally. For the numerical evaluations use (a) a single application of the trapezoidal rule, (b) Simpson’s 1y3 rule, (c) Simpson’s 3y8 rule, (d) Boole’s rule, (e) the midpoint method, (f) the 3-segment–2-point open integration formula, and (g) the 4-segment–3-point open integra- tion formula. Compute percent relative errors for the numerical results.

# 1

0 142x dx

21.8 Integrate the following function both analytically and numeri- cally. For the numerical evaluations use (a) single application of the trapezoidal rule; (b) Simpson’s 1y3 rule; (c) Simpson’s 3y8 rule; (d) multiple application of Simpson’s rules, with n 5 5; (e) Boole’s

cha9792x_ch21_603-632.indd Page 630 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 630 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 631

21.21 An 11-m beam is subjected to a load, and the shear force follows the equation

V(x) 5 5 1 0.25x2

where V is the shear force and x is length in distance along the beam. We know that V 5 dMydx, and M is the bending moment. Integration yields the relationship

M 5 Mo 1 # x

0 V dx

If Mo is zero and x 5 11, calculate M using (a) analytical integration, (b) multiple-application trapezoidal rule, and (c) multiple-application Simpson’s rules. For (b) and (c) use 1-m increments. 21.22 The work produced by a constant temperature, pressure- volume thermodynamic process can be computed as

W 5 # p dV where W is work, p is pressure, and V is volume. Using a combi- nation of the trapezoidal rule, Simpson’s 1y3 rule, and Simpson’s 3y8 rule, use the following data to compute the work in kJ (kJ 5 kN ? m):

Pressure (kPa) 336 294.4 266.4 260.8 260.5 249.6 193.6 165.6

Volume (m3) 0.5 2 3 4 6 8 10 11

21.23 Determine the distance traveled for the following data:

t, min 1 2 3.25 4.5 6 7 8 9 9.5 10

v, m/s 5 6 5.5 7 8.5 8 6 7 7 5

(a) Use the trapezoidal rule, (b) the best combination of the trape- zoidal and Simpson’s rules, and (c) analytically integrating second- and third-order polynomials determined by regression. 21.24 The total mass of a variable density rod is given by

m 5 # L

0 r(x)Ac(x) dx

where m 5 mass, r (x) 5 density, Ac(x) 5 cross-sectional area, x 5 distance along the rod, and L 5 the total length of the rod. The fol- lowing data have been measured for a 10-m length rod. Determine the mass in kilograms to the best possible accuracy.

x, m 0 2 3 4 6 8 10

r, g/cm3 4.00 3.95 3.89 3.80 3.60 3.41 3.30

Ac, cm2 100 103 106 110 120 133 150

21.25 A transportation engineering study requires that you deter- mine the number of cars that pass through an intersection traveling

(a) analytically; (b) using a multiple-application trapezoidal rule, with n 5 2; and (c) using single applications of Simpson’s 1y3 rule. For (b) and (c), compute the percent relative error (et). 21.15 Evaluate the following triple integral (a) analytically and (b) using single applications of Simpson’s 1y3 rule. For (b) compute the percent relative error (et).

# 2

22 #

2

0 #

1

23 (x3 2 3yz) dx dy dz

21.16 Develop a user-friendly computer program for the multiple- application trapezoidal rule based on Fig. 21.9. Test your program by duplicating the computation from Example 21.2. 21.17 Develop a user-friendly computer program for the multiple- application version of Simpson’s rule based on Fig. 21.13c. Test it by duplicating the computations from Example 21.5. 21.18 Develop a user-friendly computer program for integrating unequally spaced data based on Fig. 21.15b. Test it by duplicating the computation from Example 21.8. 21.19 The following data was collected for a cross-section of a river (y 5 distance from bank, H 5 depth, and U 5 velocity):

y, m 0 1 3 5 7 8 9 10

H, m 0 1 1.5 3 3.5 3.2 2 0

U, m/s 0 0.1 0.12 0.2 0.25 0.3 0.15 0

Use numerical integration to compute the (a) average depth, (b) cross-sectional area, (c) average velocity, and (d) the fl ow rate. Note that the cross-sectional area (Ac) and the fl ow rate (Q) can be computed as

Ac 5 # y

0 H(y) dy       Q 5 #

y

0 H(y)U(y) dy

21.20 The outfl ow concentration from a reactor is measured at a number of times over a 24-hr period:

t, hr 0 1 5.5 10 12 14 16 18 20 24

c, mg/L 1 1.5 2.3 2.1 4 5 5.5 5 3 1.2

The fl ow rate for the outfl ow in m3/s can be computed with the following equation:

Q(t) 5 20 1 10 sin a2p 24

(t 2 10)b Use the best numerical integration method to determine the fl ow- weighted average concentration leaving the reactor over the 24-hr period,

c 5 # t 0

Q(t)c(t)dt

# t 0 Q(t)dt

cha9792x_ch21_603-632.indd Page 631 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 631 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

632 NEWTON-COTES INTEGRATION FORMULAS

during morning rush hour. You stand at the side of the road and count the number of cars that pass every 4 minutes at several times as tabulated below. Use the best numerical method to determine (a) the total number of cars that pass between 7:30 and 9:15, and (b) the rate of cars going through the intersection per minute. (Hint: Be careful with units.)

Time (hr) 7:30 7:45 8:00 8:15 8:45 9:15

Rate (cars per 4 min) 18 24 14 24 21 9

21.26 Determine the average value for the data in Fig. P21.26. Perform the integral needed for the average in the order shown by the following equation:

I 5 # xn

x0

c # ym

y0 f(x, y)dy d dx

FIGURE P21.26

–8

–3

–1

4

7

10

–8 4

2

0 0 4 8 12

–4

–2

–6

1

4

x

y

cha9792x_ch21_603-632.indd Page 632 29/10/13 9:46 PM F-468 cha9792x_ch21_603-632.indd Page 632 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22 C H A P T E R 22

633

Integration of Equations

In the introduction to Part Six, we noted that functions to be integrated numerically will typically be of two forms: a table of values or a function. The form of the data has an important infl uence on the approaches that can be used to evaluate the integral. For tabulated information, you are limited by the number of points that are given. In contrast, if the function is available, you can generate as many values of f(x) as are required to attain acceptable accuracy (recall Fig. PT6.7). This chapter is devoted to three techniques that are expressly designed to analyze cases where the function is given. Each capitalizes on the ability to generate function values to develop effi cient schemes for numerical integration. The fi rst is based on Richardson’s ex- trapolation, which is a method for combining two numerical integral estimates to obtain a third, more accurate value. The computational algorithm for implementing Richardson’s extrapolation in a highly effi cient manner is called Romberg integration. This technique is recursive and can be used to generate an integral estimate within a prespecifi ed error tolerance. The second method, adaptive integration, is based on dividing the integration interval into successively more refi ned subintervals in a recursive fashion. Thus, more refi ned spacing is employed where the function varies rapidly and coarser spacing used where the function varies slowly in order to attain a desired global accuracy with the least computational effort. The third method is called Gauss quadrature. Recall that, in the last chapter, values of f(x) for the Newton-Cotes formulas were determined at specifi ed values of x. For example, if we used the trapezoidal rule to determine an integral, we were constrained to take the weighted average of f(x) at the ends of the interval. Gauss-quadrature formulas employ x values that are positioned between a and b in such a manner that a much more accurate integral estimate results. In addition to these two standard techniques, we devote a fi nal section to the evaluation of improper integrals. In this discussion, we focus on integrals with infi nite limits and show how a change of variable and open integration formulas prove useful for such cases.

22.1 NEWTON-COTES ALGORITHMS FOR EQUATIONS

In Chap. 21, we presented algorithms for multiple-application versions of the trapezoidal rule and Simpson’s rules. Although these pseudocodes can certainly be used to analyze equations, in our effort to make them compatible with either data or functions, they could not exploit the convenience of the latter.

cha9792x_ch22_633-654.indd Page 633 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 633 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

634 INTEGRATION OF EQUATIONS

Figure 22.1 shows pseudocodes that are expressly designed for cases where the func- tion is analytical. In particular, notice that neither the independent nor the dependent variable values are passed into the function via its argument as was the case for the codes in Chap. 21. For the independent variable x, the integration interval (a, b) and the number of segments are passed. This information is then employed to generate equispaced values of x within the function. For the dependent variable, the function values in Fig. 22.1 are computed using calls to the function being analyzed, f(x). We developed single-precision programs based on these pseudocodes to analyze the effort involved and the errors incurred as we progressively used more segments to estimate the integral of a simple function. For an analytical function, the error equa- tions [Eqs. (21.13) and (21.19)] indicate that increasing the number of segments n will result in more accurate integral estimates. This observation is borne out by Fig. 22.2, which is a plot of true error versus n for the integral of f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5. Notice how the error drops as n increases. However, also notice that at large values of n, the error starts to increase as round-off errors begin to dominate. Also observe that a very large number of function evaluations (and, hence, computational effort) is required to attain high levels of accuracy. As a consequence of these shortcomings, the multiple-application trapezoidal rule and Simpson’s rules are sometimes inadequate for problem contexts where high effi ciency and low errors are needed.

22.2 ROMBERG INTEGRATION

Romberg integration is one technique that is designed to attain effi cient numerical inte- grals of functions. It is quite similar to the techniques discussed in Chap. 21 in the sense that it is based on successive application of the trapezoidal rule. However, through math- ematical manipulations, superior results are attained for less effort.

(a) FUNCTION TrapEq (n, a, b) h 5 (b 2 a) / n x 5 a sum 5 f(x) DOFOR i 5 1, n 2 1 x 5 x 1 h sum 5 sum 1 2 * f(x) END DO sum 5 sum 1 f(b) TrapEq 5 (b 2 a) * sum / (2 * n) END TrapEq

(b) FUNCTION SimpEq (n, a, b) h 5 (b 2 a) / n x 5 a sum 5 f(x) DOFOR i 5 1, n 2 2, 2 x 5 x 1 h sum 5 sum 1 4 * f(x) x 5 x 1 h sum 5 sum 1 2 * f(x) END DO x 5 x 1 h sum 5 sum 1 4 * f(x) sum 5 sum 1 f(b) SimpEq 5 (b 2 a) * sum /(3 * n) END SimpEq

FIGURE 22.1 Algorithms for multiple applica- tions of the (a) trapezoidal and (b) Simpson’s 1/3 rules, where the function is available.

cha9792x_ch22_633-654.indd Page 634 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 634 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.2 ROMBERG INTEGRATION 635

22.2.1 Richardson’s Extrapolation

Recall that, in Sec. 10.3.3, we used iterative refi nement to improve the solution of a set of simultaneous linear equations. Error-correction techniques are also available to improve the results of numerical integration on the basis of the integral estimates themselves. Generally called Richardson’s extrapolation, these methods use two estimates of an in- tegral to compute a third, more accurate approximation. The estimate and error associated with a multiple-application trapezoidal rule can be represented generally as

I 5 I(h) 1 E(h)

where I 5 the exact value of the integral, I(h) 5 the approximation from an n-segment application of the trapezoidal rule with step size h 5 (b 2 a)yn, and E(h) 5 the trunca- tion error. If we make two separate estimates using step sizes of h1 and h2 and have exact values for the error,

I(h1) 1 E(h1) 5 I(h2) 1 E(h2) (22.1)

FIGURE 22.2 Absolute value of the true per- cent relative error versus number of segments for the determina- tion of the integral of f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5, evaluated from a 5 0 to b 5 0.8 using the multiple- application trapezoidal rule and the multiple-application Simpson’s 1/3 rule. Note that both results indicate that for a large number of segments, round-off errors limit precision.

1 4 16 64 256 1024 4096 16384 2 8 32 128

Segments

Simpson’s 1/3 rule

Limit of precision

Limit of precision

Trapezoidal rule

T ru

e p

er ce

n t

re la

ti ve

e rr

o r

512 2048 8192

100

10

1

10– 1

10– 2

10– 3

10– 4

10– 5

10– 6

cha9792x_ch22_633-654.indd Page 635 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 635 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

636 INTEGRATION OF EQUATIONS

Now recall that the error of the multiple-application trapezoidal rule can be represented approximately by Eq. (21.13) [with n 5 (b 2 a)yh]:

E > 2 b 2 a

12 h2 f – (22.2)

If it is assumed that f – is constant regardless of step size, Eq. (22.2) can be used to determine that the ratio of the two errors will be

E(h1)

E(h2) >

h21 h22

(22.3)

This calculation has the important effect of removing the term f – from the computation. In so doing, we have made it possible to utilize the information embodied by Eq. (22.2) without prior knowledge of the function’s second derivative. To do this, we rearrange Eq. (22.3) to give

E(h1) > E(h2)ah1h2b 2

which can be substituted into Eq. (22.1):

I(h1) 1 E(h2)ah1h2b 2

> I(h2) 1 E(h2)

which can be solved for

E(h2) > I(h1) 2 I(h2)

1 2 (h1yh2) 2

Thus, we have developed an estimate of the truncation error in terms of the integral estimates and their step sizes. This estimate can then be substituted into

I 5 I(h2) 1 E(h2)

to yield an improved estimate of the integral:

I > I(h2) 1 1

(h1yh2) 2 2 1

[I(h2) 2 I(h1) ] (22.4)

It can be shown (Ralston and Rabinowitz, 1978) that the error of this estimate is O(h4). Thus, we have combined two trapezoidal rule estimates of O(h2) to yield a new estimate of O(h4). For the special case where the interval is halved (h2 5 h1y2), this equation becomes

I > I(h2) 1 1

22 2 1 [I(h2) 2 I(h1) ]

or, collecting terms,

I > 4

3 I(h2) 2

1

3 I(h1) (22.5)

EXAMPLE 22.1 Error Corrections of the Trapezoidal Rule

Problem Statement. In the previous chapter (Example 21.1 and Table 21.1), we used a variety of numerical integration methods to evaluate the integral of f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5 from a 5 0 to b 5 0.8. For example, single and multiple

cha9792x_ch22_633-654.indd Page 636 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 636 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.2 ROMBERG INTEGRATION 637

applications of the trapezoidal rule yielded the following results:

Segments h Integral Et, %

1 0.8 0.1728 89.5 2 0.4 1.0688 34.9 4 0.2 1.4848 9.5

Use this information along with Eq. (22.5) to compute improved estimates of the integral.

Solution. The estimates for one and two segments can be combined to yield

I > 4

3 (1.0688) 2

1

3 (0.1728) 5 1.367467

The error of the improved integral is Et 5 1.640533 2 1.367467 5 0.273067 (et 5 16.6%), which is superior to the estimates upon which it was based. In the same manner, the estimates for two and four segments can be combined to give

I > 4

3 (1.4848) 2

1

3 (1.0688) 5 1.623467

which represents an error of Et 5 1.640533 2 1.623467 5 0.017067 (et 5 1.0%).

Equation (22.4) provides a way to combine two applications of the trapezoidal rule with error O(h2) to compute a third estimate with error O(h4). This approach is a subset of a more general method for combining integrals to obtain improved estimates. For instance, in Example 22.1, we computed two improved integrals of O(h4) on the basis of three trapezoidal rule estimates. These two improved estimates can, in turn, be com- bined to yield an even better value with O(h6). For the special case where the original trapezoidal estimates are based on successive halving of the step size, the equation used for O(h6) accuracy is

I > 16

5 Im 2

1

15 Il (22.6)

where Im and Il are the more and less accurate estimates, respectively. Similarly, two O(h6) results can be combined to compute an integral that is O(h8) using

I > 64

63 Im 2

1

63 Il (22.7)

EXAMPLE 22.2 Higher-Order Error Correction of Integral Estimates

Problem Statement. In Example 22.1, we used Richardson’s extrapolation to compute two integral estimates of O(h4). Utilize Eq. (22.6) to combine these estimates to compute an integral with O(h6).

cha9792x_ch22_633-654.indd Page 637 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 637 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

638 INTEGRATION OF EQUATIONS

Solution. The two integral estimates of O(h4) obtained in Example 22.1 were 1.367467 and 1.623467. These values can be substituted into Eq. (22.6) to yield

I 5 16

15 (1.623467) 2

1

15 (1.367467) 5 1.640533

which is the correct answer to the seven signifi cant fi gures that are carried in this example.

22.2.2 The Romberg Integration Algorithm

Notice that the coeffi cients in each of the extrapolation equations [Eqs. (22.5), (22.6), and (22.7)] add up to 1. Thus, they represent weighting factors that, as accuracy in- creases, place relatively greater weight on the superior integral estimate. These for- mulations can be expressed in a general form that is well-suited for computer implementation:

Ij, k > 4k21 Ij11, k21 2 Ij, k21

4k21 2 1 (22.8)

where Ij11, k21 and Ij, k21 5 the more and less accurate integrals, respectively, and Ij, k 5 the improved integral. The index k signifi es the level of the integration, where k 5 1 cor- responds to the original trapezoidal rule estimates, k 5 2 corresponds to O(h4), k 5 3 to O(h6), and so forth. The index j is used to distinguish between the more ( j 1 1) and the less ( j) accurate estimates. For example, for k 5 2 and j 5 1, Eq. (22.8) becomes

I1, 2 > 4I2,1 2 I1,1

3

which is equivalent to Eq. (22.5). The general form represented by Eq. (22.8) is attributed to Romberg, and its systematic application to evaluate integrals is known as Romberg integration. Figure 22.3 is a

FIGURE 22.3 Graphical depiction of the sequence of integral estimates generated using Romberg integration. (a) First iteration. (b) Second iteration. (c) Third iteration.

O(h2) O(h4) O(h6) O(h8)

(a) 0.172800 1.367467 1.068800

(b) 0.172800 1.367467 1.640533 1.068800 1.623467 1.484800

(c) 0.172800 1.367467 1.640533 1.640533 1.068800 1.623467 1.640533 1.484800 1.639467 1.600800

cha9792x_ch22_633-654.indd Page 638 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 638 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.2 ROMBERG INTEGRATION 639

graphical depiction of the sequence of integral estimates generated using this approach. Each matrix corresponds to a single iteration. The fi rst column contains the trapezoidal rule evaluations that are designated Ij,1, where j 5 1 is for a single-segment application (step size is b 2 a), j 5 2 is for a two-segment application [step size is (b 2 a)y2], j 5 3 is for a four-segment application [step size is (b 2 a)y4], and so forth. The other columns of the matrix are generated by systematically applying Eq. (22.8) to obtain successively better estimates of the integral. For example, the fi rst iteration (Fig. 22.3a) involves computing the one- and two- segment trapezoidal rule estimates (I1,1 and I2,1). Equation (22.8) is then used to compute the element I1,2 5 1.367467, which has an error of O(h

4). Now, we must check to determine whether this result is adequate for our needs. As in other approximate methods in this book, a termination, or stopping, criterion is re- quired to assess the accuracy of the results. One method that can be employed for the present purposes is [Eq. (3.5)]

Zea Z 5 ` I1, k 2 I2, k21I1,k ` 100% (22.9) where ea 5 an estimate of the percent relative error. Thus, as was done previously in other iterative processes, we compare the new estimate with a previous value. When the change between the old and new values as represented by ea is below a prespecifi ed error criterion es, the computation is terminated. For Fig. 22.3a, this evaluation indicates an 21.8 percent change over the course of the fi rst iteration. The object of the second iteration (Fig. 22.3b) is to obtain the O(h6) estimate—I1,3. To do this, an additional trapezoidal rule estimate, I3,1 5 1.4848, is determined. Then it is combined with I2,1 using Eq. (22.8) to generate I2,2 5 1.623467. The result is, in turn, combined with I1,2 to yield I1,3 5 1.640533. Equation (22.9) can be applied to determine that this result represents a change of 1.0 percent when compared with the previous result I1,2. The third iteration (Fig. 22.3c) continues the process in the same fashion. In this case, a trapezoidal estimate is added to the fi rst column, and then Eq. (22.8) is applied to com- pute successively more accurate integrals along the lower diagonal. After only three it- erations, because we are evaluating a fi fth-order polynomial, the result (I1,4 5 1.640533) is exact. Romberg integration is more effi cient than the trapezoidal rule and Simpson’s rules discussed in Chap. 21. For example, for determination of the integral as shown in Fig. 22.1, Simpson’s 1y3 rule would require a 256-segment application to yield an estimate of 1.640533. Finer approximations would not be possible because of round-off error. In contrast, Romberg integration yields an exact result (to seven signifi cant fi gures) based on combining one-, two-, four-, and eight-segment trapezoidal rules; that is, with only 15 function evaluations! Figure 22.4 presents pseudocode for Romberg integration. By using loops, this algorithm implements the method in an effi cient manner. Romberg integration is designed for cases where the function to be integrated is known. This is because knowledge of the function permits the evaluations required for the initial implementations of the trapezoidal rule. Tabulated data are rarely in the form needed to make the necessary successive halvings.

cha9792x_ch22_633-654.indd Page 639 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 639 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

640 INTEGRATION OF EQUATIONS

22.3 ADAPTIVE QUADRATURE

Although Romberg integration is more effi cient than the composite Simpson’s 1y3 rule, both use equally spaced points. This global perspective ignores the fact that many functions have regions of high variability along with other sections where change is gradual. Adaptive quadrature methods remedy this situation by adjusting the step size so that small intervals are used in regions of rapid variations and larger intervals are used where the function changes gradually. Most of these techniques are based on applying the composite Simpson’s 1y3 rule to subintervals in a fashion that is very similar to the way in which the composite trapezoidal rule was used in Richardson extrapolation. That is, the 1y3 rule is applied at two levels of refi nement and the difference between these two levels is used to estimate the truncation error. If the truncation error is acceptable, no further refi nement is required and the integral estimate for the subinterval is deemed acceptable. If the error estimate is too large, the step size is refi ned and the process repeated until the error falls to acceptable levels. The total integral is then computed as the summation of the integral estimates for the subintervals. The theoretical basis of the approach can be illustrated for an interval x 5 a to x 5 b with a width of h1 5 b 2 a. A fi rst estimate of the integral can be estimated with Simpson’s 1y3 rule,

I(h1) 5 h1 6

( f(a) 1 4 f(c) 1 f(b)) (22.10)

where c 5 (a 1 b)y2.

FIGURE 22.4 Pseudocode for Romberg integration that uses the equal-size-segment version of the trapezoidal rule from Fig. 22.1.

FUNCTION Romberg (a, b, maxit, es) LOCAL I(10, 10) n 5 1 I1,1 5 TrapEq(n, a, b) iter 5 0 DO iter 5 iter 1 1 n 5 2iter

Iiter11,1 5 TrapEq(n, a, b) DOFOR k 5 2, iter 1 1 j 5 2 1 iter 2 k Ij,k 5 (4

k21 * Ij11,k21 2 Ij,k21) y (4 k21 2 1)

END DO ea 5 ABS((I1,iter11 2 I2,iter) y I1,iter11) * 100 IF (iter $ maxit OR ea # es) EXIT END DO Romberg 5 I1,iter11 END Romberg

cha9792x_ch22_633-654.indd Page 640 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 640 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.3 ADAPTIVE QUADRATURE 641

As in Richardson extrapolation, a more refi ned estimate can be obtained by halving the step size. That is, by applying the multiple-application Simpson’s 1y3 rule with n 5 4,

I(h2) 5 h2 6

( f(a) 1 4 f(d) 1 2 f(c) 1 4 f(e) 1 f(b)) (22.11)

where d 5 (a 1 c)y2, e 5 (c 1 b)y2, and h2 5 h1y2. Because both I(h1) and I(h2) are estimates of the same integral, their difference provides a measure of the error. That is,

E > I(h2) 2 I(h1) (22.12)

In addition, the estimate and error associated with either application can be represented generally as

I 5 I(h) 1 E(h) (22.13)

where I 5 the exact value of the integral, I(h) 5 the approximation from an n-segment application of the Simpson’s 1y3 rule with step size h 5 (b 2 a)yn, and E(h) 5 the corresponding truncation error. Using an approach similar to Richardson extrapolation, we can derive an estimate for the error of the more refi ned estimate, I(h2), as a function of the difference between the two integral estimates,

E(h2) 5 1

15 [I(h2) 2 I(h1) ] (22.14)

The error can then be added to I(h2) to generate an even better estimate

I 5 I(h2) 1 1

15 [I(h2) 2 I(h1) ] (22.15)

This result is equivalent to Boole’s rule. The equations developed above can now be combined into an effi cient algorithm. Figure 22.5 presents pseudocode for such an algorithm that is based on a MATLAB software M-fi le developed by Cleve Moler (2005). The function consists of a main calling function, quadapt, along with a recursive function, qstep, that actually performs the integration. As set up in Fig. 22.5, both qadapt and qstep must have access to another function, f, that evaluates the integrand. The main calling function, quadapt, is passed the integration limits, a and b. After setting the tolerance, the function evaluations required for the initial application of Simpson’s 1y3 rule (Eq. 22.10) are computed. These values along with the integration limits are then passed to qstep. Within qstep, the remaining step sizes and function values are deter- mined and the two integral estimates (Eqs. 22.10 and 22.11) are computed. At this point, the error is estimated as the absolute difference between the integral estimates. Depending on the value of the error, two things can then happen:

1) If the error is less than or equal to the tolerance, Boole’s rule is generated, the func- tion terminates and the result is returned.

2) If the error is larger than the tolerance, qstep is invoked twice to evaluate each of the two subintervals of the current call.

cha9792x_ch22_633-654.indd Page 641 09/12/13 9:11 AM F-468 cha9792x_ch22_633-654.indd Page 641 09/12/13 9:11 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

642 INTEGRATION OF EQUATIONS

The two recursive calls in the second step represent the real beauty of this algorithm. They just keep subdividing until the tolerance is met. Once this occurs, their results are passed back up the recursive path, combining with the other integral estimates along the way. The process ends when the fi nal call is satisfi ed and the total integral is evaluated and returned to the main calling function. It should be stressed that the algorithm in Fig. 22.5 is a stripped down version of the quad function which is the professional quadrature function employed in MATLAB. Thus, it does not guard against failure such as cases where integrals do not exist. Nevertheless, it works just fi ne for many applications, and certainly serves to illustrate how adaptive quadrature works.

22.4 GAUSS QUADRATURE

In Chap. 21, we studied the group of numerical integration or quadrature formulas known as the Newton-Cotes equations. A characteristic of these formulas (with the exception of the special case of Sec. 21.3) was that the integral estimate was based on evenly spaced function values. Consequently, the location of the base points used in these equations was predetermined or fi xed.

FUNCTION quadapt(a, b) (main calling function) tol 5 0.000001 c 5 (a 1 b)/2 (initialization) fa 5 f(a) fc 5 f(c) fb 5 f(b) quadapt 5 qstep(a, b, tol, fa, fc, fb) END quadapt

FUNCTION qstep(a, b, tol, fa, fc, fb) (recursive function) h1 5 b 2 a h2 5 h1/2 c 5 (a 1 b)/2 fd 5 f((a 1 c)/2) fe 5 f((c 1 b)/2) I1 5 h1/6 * (fa 1 4 * fc 1 fb) (Simpson’s 1/3 rule) I2 5 h2/6 * (fa 1 4 * fd 1 2 * fc 1 4 * fe 1 fb) IF |I2 2 I1| # tol THEN (terminate after Boole’s rule) I 5 I2 1 (I2 2 I1)/15 ELSE (recursive calls if needed) Ia 5 qstep(a, c, tol, fa, fd, fc) Ib 5 qstep(c, b, tol, fc, fe, fb) I 5 Ia 1 Ib END IF qstep 5 I END qstep

FIGURE 22.5 Pseudocode for simplifi ed adap- tive quadrature algorithm based on a MATLAB M-fi le presented in Moler (2005).

cha9792x_ch22_633-654.indd Page 642 09/12/13 9:12 AM F-468 cha9792x_ch22_633-654.indd Page 642 09/12/13 9:12 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.4 GAUSS QUADRATURE 643

For example, as depicted in Fig. 22.6a, the trapezoidal rule is based on taking the area under the straight line connecting the function values at the ends of the integration interval. The formula that is used to compute this area is

I > (b 2 a) f(a) 1 f(b)

2 (22.16)

where a and b 5 the limits of integration and b 2 a 5 the width of the integration interval. Because the trapezoidal rule must pass through the end points, there are cases such as Fig. 22.6a where the formula results in a large error. Now, suppose that the constraint of fi xed base points was removed and we were free to evaluate the area under a straight line joining any two points on the curve. By positioning these points wisely, we could defi ne a straight line that would balance the positive and negative errors. Hence, as in Fig. 22.6b, we would arrive at an improved estimate of the integral. Gauss quadrature is the name for one class of techniques to implement such a strategy. The particular Gauss quadrature formulas described in this section are called Gauss-Legendre formulas. Before describing the approach, we will show how numerical integration formulas such as the trapezoidal rule can be derived using the method of undetermined coeffi cients. This method will then be employed to develop the Gauss- Legendre formulas.

FIGURE 22.6 (a) Graphical depiction of the trapezoidal rule as the area under the straight line joining fi xed end points. (b) An improved integral estimate obtained by taking the area under the straight line passing through two intermediate points. By positioning these points wisely, the positive and negative errors are balanced, and an improved integral estimate results.

f (x)

(a)

x

f (x)

(b)

x

cha9792x_ch22_633-654.indd Page 643 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 643 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

644 INTEGRATION OF EQUATIONS

22.4.1 Method of Undetermined Coeffi cients

In Chap. 21, we derived the trapezoidal rule by integrating a linear interpolating polynomial and by geometrical reasoning. The method of undetermined coeffi cients offers a third ap- proach that also has utility in deriving other integration techniques such as Gauss quadrature. To illustrate the approach, Eq. (22.16) is expressed as

I > c0 f(a) 1 c1 f(b) (22.17)

where the c’s 5 constants. Now realize that the trapezoidal rule should yield exact results when the function being integrated is a constant or a straight line. Two simple equations that represent these cases are y 5 1 and y 5 x. Both are illustrated in Fig. 22.7. Thus, the following equalities should hold:

c0 1 c1 5 # (b2a)y2

2(b2a)y2

1 dx

and

2c0 b 2 a

2 1 c1

b 2 a

2 5 #

(b2a)y2

2(b2a)y2 x dx

y

y = 1

(a)

x– (b – a) 2

b – a 2

y

y = x

(b)

x

– (b – a) 2

b – a 2

FIGURE 22.7 Two integrals that should be evaluated exactly by the trap- ezoidal rule: (a) a constant and (b) a straight line.

cha9792x_ch22_633-654.indd Page 644 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 644 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.4 GAUSS QUADRATURE 645

or, evaluating the integrals,

c0 1 c1 5 b 2 a

and

2c0 b 2 a

2 1 c1

b 2 a

2 5 0

These are two equations with two unknowns that can be solved for

c0 5 c1 5 b 2 a

2

which, when substituted back into Eq. (22.17), gives

I 5 b 2 a

2 f(a) 1

b 2 a

2 f(b)

which is equivalent to the trapezoidal rule.

22.4.2 Derivation of the Two-Point Gauss-Legendre Formula

Just as was the case for the above derivation of the trapezoidal rule, the object of Gauss quadrature is to determine the coeffi cients of an equation of the form

I > c0  f(x0) 1 c1 f(x1) (22.18)

where the c’s 5 the unknown coeffi cients. However, in contrast to the trapezoidal rule that used fi xed end points a and b, the function arguments x0 and x1 are not fi xed at the end points, but are unknowns (Fig. 22.8). Thus, we now have a total of four unknowns that must be evaluated, and consequently, we require four conditions to determine them exactly.

FIGURE 22.8 Graphical depiction of the unknown variables x0 and x1 for integration by Gauss quadrature.

f (x)

f (x0)

f (x1)

– 1 1x1 xx0

cha9792x_ch22_633-654.indd Page 645 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 645 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

646 INTEGRATION OF EQUATIONS

Just as for the trapezoidal rule, we can obtain two of these conditions by assuming that Eq. (22.18) fi ts the integral of a constant and a linear function exactly. Then, to ar- rive at the other two conditions, we merely extend this reasoning by assuming that it also fi ts the integral of a parabolic (y 5 x2) and a cubic (y 5 x3) function. By doing this, we determine all four unknowns and in the bargain derive a linear two-point integration formula that is exact for cubics. The four equations to be solved are

c0 f(x0) 1 c1  f(x1) 5 # 1

21 1 dx 5 2 (22.19)

c0 f(x0) 1 c1  f(x1) 5 # 1

21 x dx 5 0 (22.20)

c0 f(x0) 1 c1 f(x1) 5 # 1

21 x2 dx 5

2

3 (22.21)

c0 f(x0) 1 c1  f(x1) 5 # 1

21 x3 dx 5 0 (22.22)

Equations (22.19) through (22.22) can be solved simultaneously for

c0 5 c1 5 1

x0 5 2 1

23 5 20.5773503 p

x1 5 1

23 5 0.5773503 p

which can be substituted into Eq. (22.18) to yield the two-point Gauss-Legendre formula

I > f a 21 23 b 1 f a 1

23 b (22.23)

Thus, we arrive at the interesting result that the simple addition of the function values at x 5 1y13 and 21y13 yields an integral estimate that is third-order accurate. Notice that the integration limits in Eqs. (22.19) through (22.22) are from 21 to 1. This was done to simplify the mathematics and to make the formulation as general as possible. A simple change of variable can be used to translate other limits of integration into this form. This is accomplished by assuming that a new variable xd is related to the original variable x in a linear fashion, as in

x 5 a0 1 a1xd (22.24)

If the lower limit, x 5 a, corresponds to xd 5 21, these values can be substituted into Eq. (22.24) to yield

a 5 a0 1 a1(21) (22.25)

Similarly, the upper limit, x 5 b, corresponds to xd 5 1, to give

b 5 a0 1 a1(1) (22.26)

cha9792x_ch22_633-654.indd Page 646 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 646 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.4 GAUSS QUADRATURE 647

Equations (22.25) and (22.26) can be solved simultaneously for

a0 5 b 1 a

2 (22.27)

and

a1 5 b 2 a

2 (22.28)

which can be substituted into Eq. (22.24) to yield

x 5 (b 1 a) 1 (b 2 a)xd

2 (22.29)

This equation can be differentiated to give

dx 5 b 2 a

2 dxd (22.30)

Equations (22.29) and (22.30) can be substituted for x and dx, respectively, in the equation to be integrated. These substitutions effectively transform the integration interval without chang- ing the value of the integral. The following example illustrates how this is done in practice.

EXAMPLE 22.3 Two-Point Gauss-Legendre Formula

Problem Statement. Use Eq. (22.23) to evaluate the integral of

f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5

between the limits x 5 0 to 0.8. Recall that this was the same problem that we solved in Chap. 21 using a variety of Newton-Cotes formulations. The exact value of the integral is 1.640533.

Solution. Before integrating the function, we must perform a change of variable so that the limits are from 21 to 11. To do this, we substitute a 5 0 and b 5 0.8 into Eq. (22.29) to yield

x 5 0.4 1 0.4xd

The derivative of this relationship is [Eq. (22.30)]

dx 5 0.4 dxd

Both of these can be substituted into the original equation to yield

# 0.8

0

(0.2 1 25x 2 200x2 1 675x3 2 900x4 1 400x5) dx

5 # 1

21

[0.2 1 25(0.4 1 0.4xd) 2 200(0.4 1 0.4xd) 2 1 675(0.4 1 0.4xd)

3

2 900(0.4 1 0.4xd) 4 1 400(0.4 1 0.4xd)

5]0.4 dxd

Therefore, the right-hand side is in the form that is suitable for evaluation using Gauss quadrature. The transformed function can be evaluated at 21y13 to be equal to 0.516741

cha9792x_ch22_633-654.indd Page 647 29/10/13 9:46 PM F-468 cha9792x_ch22_633-654.indd Page 647 29/10/13 9:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

648 INTEGRATION OF EQUATIONS

and at 1y13 to be equal to 1.305837. Therefore, the integral according to Eq. (22.23) is

I > 0.516741 1 1.305837 5 1.822578

which represents a percent relative error of 211.1 percent. This result is comparable in magnitude to a four-segment application of the trapezoidal rule (Table 21.1) or a single application of Simpson’s 1y3 and 3y8 rules (Examples 21.4 and 21.6). This latter result is to be expected because Simpson’s rules are also third-order accurate. However, because of the clever choice of base points, Gauss quadrature attains this accuracy on the basis of only two function evaluations.

22.4.3 Higher-Point Formulas

Beyond the two-point formula described in the previous section, higher-point versions can be developed in the general form

I > c0 f(x0) 1 c1  f(x1) 1 p 1 cn21  f(xn21) (22.31)

where n 5 the number of points. Values for c’s and x’s for up to and including the six- point formula are summarized in Table 22.1.

TABLE 22.1 Weighting factors c and function arguments x used in Gauss-Legendre formulas.

Weighting Function Truncation Points Factors Arguments Error

2 c0 5 1.0000000 x0 5 20.577350269 >f (4)(j) c1 5 1.0000000 x1 5 0.577350269

3 c0 5 0.5555556 x0 5 20.774596669 >f (6)(j) c1 5 0.8888889 x1 5 0.0 c2 5 0.5555556 x2 5 0.774596669

4 c0 5 0.3478548 x0 5 20.861136312 >f (8)(j) c1 5 0.6521452 x1 5 20.339981044 c2 5 0.6521452 x2 5 0.339981044 c3 5 0.3478548 x3 5 0.861136312

5 c0 5 0.2369269 x0 5 20.906179846 >f (10)(j) c1 5 0.4786287 x1 5 20.538469310 c2 5 0.5688889 x2 5 0.0 c3 5 0.4786287 x3 5 0.538469310 c4 5 0.2369269 x4 5 0.906179846

6 c0 5 0.1713245 x0 5 20.932469514 >f (12)(j) c1 5 0.3607616 x1 5 20.661209386 c2 5 0.4679139 x2 5 20.238619186 c3 5 0.4679139 x3 5 0.238619186 c4 5 0.3607616 x4 5 0.661209386 c5 5 0.1713245 x5 5 0.932469514

cha9792x_ch22_633-654.indd Page 648 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 648 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.4 GAUSS QUADRATURE 649

EXAMPLE 22.4 Three-Point Gauss-Legendre Formula

Problem Statement. Use the three-point formula from Table 22.1 to estimate the integral for the same function as in Example 22.3.

Solution. According to Table 22.1, the three-point formula is

I 5 0.5555556 f(20.7745967) 1 0.8888889 f(0) 1 0.5555556 f(0.7745967)

which is equal to

I 5 0.2813013 1 0.8732444 1 0.4859876 5 1.640533

which is exact.

Because Gauss quadrature requires function evaluations at nonuniformly spaced points within the integration interval, it is not appropriate for cases where the function is unknown. Thus, it is not suited for engineering problems that deal with tabulated data. However, where the function is known, its effi ciency can be a decided advantage. This is particularly true when numerous integral evaluations must be performed.

EXAMPLE 22.5 Applying Gauss Quadrature to the Falling Parachutist Problem

Problem Statement. In Example 21.3, we used the multiple-application trapezoidal rule to evaluate

d 5 gm

c # 10

0

[1 2 e2(cym)t] dt

where g 5 9.8, c 5 12.5, and m 5 68.1. The exact value of the integral was determined by calculus to be 289.4351. Recall that the best estimate obtained using a 500-segment trapezoidal rule was 289.4348 with an uetu > 1.15 3 10

24 percent. Repeat this computa- tion using Gauss quadrature.

Solution. After modifying the function, the following results are obtained:

Two-point estimate 5 290.0145

Three-point estimate 5 289.4393

Four-point estimate 5 289.4352

Five-point estimate 5 289.4351

Six-point estimate 5 289.4351

Thus, the fi ve- and six-point estimates yield results that are exact to seven signifi cant fi gures.

22.4.4 Error Analysis for Gauss Quadrature

The error for the Gauss-Legendre formulas is specifi ed generally by (Carnahan et al., 1969)

Et 5 22n13[ (n 1 1)!]4

(2n 1 3)[(2n 1 2)!]3 f (2n12)(j) (22.32)

cha9792x_ch22_633-654.indd Page 649 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 649 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

650 INTEGRATION OF EQUATIONS

where n 5 the number of points minus one and f (2n12)(j) 5 the (2n 1 2)th derivative of the function after the change of variable with j located somewhere on the interval from 21 to 1. Comparison of Eq. (22.32) with Table 21.2 indicates the superiority of Gauss quadrature to Newton-Cotes formulas, provided the higher-order derivatives do not increase substantially with increasing n. Problem 22.8 at the end of this chapter illustrates a case where the Gauss-Legendre formulas perform poorly. In these situations, the multiple-application Simpson’s rule or Romberg integration would be preferable. However, for many functions confronted in engineering practice, Gauss quadrature provides an effi cient means for evaluating integrals.

22.5 IMPROPER INTEGRALS

To this point, we have dealt exclusively with integrals having fi nite limits and bounded integrands. Although these types are commonplace in engineering, there will be times when improper integrals must be evaluated. In this section, we will focus on one type of improper integral—that is, one with a lower limit of 2q or an upper limit of 1q. Such integrals usually can be evaluated by making a change of variable that trans- forms the infi nite range to one that is fi nite. The following identity serves this purpose and works for any function that decreases toward zero at least as fast as lyx2 as x approaches infi nity:

# b

a f(x) dx 5 #

1ya

1yb

1

t2 f a1

t b dt (22.33)

for ab . 0. Therefore, it can be used only when a is positive and b is q or when a is 2q and b is negative. For cases where the limits are from 2q to a positive value or from a negative value to q, the integral can be implemented in two steps. For example,

# b

2q f(x) dx 5 #

2A

2q f(x) dx 1 #

b

2A f(x) dx (22.34)

where 2A is chosen as a suffi ciently large negative value so that the function has begun to approach zero asymptotically at least as fast as lyx2. After the integral has been divided into two parts, the fi rst can be evaluated with Eq. (22.33) and the second with a Newton- Cotes closed formula such as Simpson’s 1y3 rule. One problem with using Eq. (22.33) to evaluate an integral is that the transformed function will be singular at one of the limits. The open integration formulas can be used to circumvent this dilemma as they allow evaluation of the integral without employing data at the end points of the integration interval. To allow the maximum fl exibility, a multiple-application version of one of the open formulas from Table 21.4 is required. Multiple-application versions of the open formulas can be concocted by using closed formulas for the interior segments and open formulas for the ends. For example, the multiple-segment trapezoidal rule and the midpoint rule can be combined to give

# xn

x0 f(x) dx 5 h c 3

2 f(x1) 1 a

n22

i52 f(xi) 1

3

2 f(xn21) d

cha9792x_ch22_633-654.indd Page 650 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 650 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

22.5 IMPROPER INTEGRALS 651

In addition, semiopen formulas can be developed for cases where one or the other end of the interval is closed. For example, a formula that is open at the lower limit and closed at the upper limit is given as

# xn

x0 f(x) dx 5 h c 3

2 f(x1) 1 a

n21

i52 f(xi) 1

1

2 f(xn) d

Although these relationships can be used, a preferred formula is (Press et al., 2007)

# xn

x0 f(x) dx 5 h[ f(x1y2) 1 f(x3y2) 1 p 1 f(xn23y2) 1 f(xn21y2) ] (22.35)

which is called the extended midpoint rule. Notice that this formula is based on limits of integration that are hy2 after and before the fi rst and last data points (Fig. 22.9).

EXAMPLE 22.6 Evaluation of an Improper Integral

Problem Statement. The cumulative normal distribution is an important formula in statistics (see Fig. 22.10):

N(x) 5 # x

2q

1

22p e2x

2y2 dx (E22.6.1)

where x 5 (y 2 y)ysy is called the normalized standard deviate. It represents a change of variable to scale the normal distribution so that it is centered on zero and the distance along the abscissa is measured in multiples of the standard deviation (Fig. 22.10b). Equation (E22.6.1) represents the probability that an event will be less than x. For example, if x 5 1, Eq. (E22.6.1) can be used to determine that the probability that an event will occur that is less than one standard deviation above the mean is N(1) 5 0.8413. In other words, if 100 events occur, approximately 84 will be less than the mean plus one standard deviation. Because Eq. (E22.6.1) cannot be evaluated in a simple functional form, it is solved numerically and listed in statistical tables. Use Eq. (22.34) in conjunction with Simpson’s 1y3 rule and the extended midpoint rule to determine N(1) numerically.

Solution. Equation (E22.6.1) can be reexpressed in terms of Eq. (22.34) as

N(x) 5 1

12p a #

22

2q e2x

2y2 dx 1 # 1

22

e2x 2y2 dxb

The fi rst integral can be evaluated by applying Eq. (22.33) to give

# 22

2q e2x

2y2 dx 5 #

0

21y2

1

t2 e21y(2t

2) dt

x0

x1/2 x3/2 x5/2

xn

xn – 5/2 xn – 3/2 xn – 1/2FIGURE 22.9 Placement of data points rela- tive to integration limits for the extended midpoint rule.

cha9792x_ch22_633-654.indd Page 651 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 651 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

652 INTEGRATION OF EQUATIONS

Then the extended midpoint rule with h 5 1y8 can be employed to estimate

# 0

21y2

1

t2 e21y(2t

2) dt > 1

8 [ f(x27y16) 1 f(x25y16) 1 f(x23y16) 1 f(x21y16) ]

5 1

8 [0.3833 1 0.0612 1 0 1 0] 5 0.0556

FIGURE 22.10 (a) The normal distribution, (b) the transformed abscissa in terms of the standardized normal deviate, and (c) the cumulative normal distribution. The shaded area in (a) and the point in (c) represent the probability that a random event will be less than the mean plus one standard deviation.

(a) y

x

y– – 2sy y – + 2syy

–y– – sy y – + sy

(b) – 3 – 2 – 1 3210

x

N(x)

(c) – 3 – 2 – 1 3210

0.5

1

N(x) = � e– x2/2 dx 1 – �

1 2�

N(x) = � e– x2/2 dx 1 – �

1 2�

cha9792x_ch22_633-654.indd Page 652 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 652 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 653

Simpson’s 1y3 rule with h 5 0.5 can be used to estimate the second integral as

# 1

22 e2x

2y2 dx

5 [1 2 (22)] 0.1353 1 4(0.3247 1 0.8825 1 0.8825) 1 2(0.6065 1 1) 1 0.6065

3(6) 5 2.0523

Therefore, the fi nal result can be computed as

N(1) > 1

22p (0.0556 1 2.0523) 5 0.8409

which represents an error of et 5 0.046 percent.

The foregoing computation can be improved in a number of ways. First, higher- order formulas could be used. For example, a Romberg integration could be employed. Second, more points could be used. Press et al. (2007) explore both options in depth. Aside from infi nite limits, there are other ways in which an integral can be improper. Common examples include cases where the integral is singular at either the limits or at a point within the integral. Press et al. (2007) provide a nice discussion of ways to handle these situations.

PROBLEMS

22.1 Use order of h8 Romberg integration to evaluate

# 3

0 xe2x dx

Compare ea and et. 22.2 Use Romberg integration to evaluate

I 5 # 2

1 ax 1 1

x b2 dx

to an accuracy of es 5 0.5% based on Eq. (22.9). Your results should be presented in the form of Fig. 22.3. Use the analytical solution of the integral to determine the percent relative error of the result obtained with Romberg integration. Check that et is less than the stopping criterion es. 22.3 Use Romberg integration to evaluate

# 2

0

ex sin x

1 1 x2 dx

to an accuracy of es 5 0.5%. Your results should be presented in the form of Fig. 22.3.

22.4 Obtain an estimate of the integral from Prob. 22.2, but using two-, three-, and four-point Gauss-Legendre formulas. Compute et for each case on the basis of the analytical solution. 22.5 Obtain an estimate of the integral from Prob. 22.1, but using two-, three-, and four-point Gauss-Legendre formulas. Compute et for each case on the basis of the analytical solution. 22.6 Obtain an estimate of the integral from Prob. 22.3 using the fi ve-point Gauss-Legendre formula. 22.7 Perform the computation in Examples 21.3 and 22.5 for the falling parachutist, but use Romberg integration (es 5 0.05%). 22.8 Employ two- through six-point Gauss-Legendre formulas to solve

# 3

23

1

1 1 x2 dx

Interpret your results in light of Eq. (22.32). 22.9 Use numerical integration to evaluate the following:

(a) # q

2

dx

x(x 1 2) (b) #

q

0 e2y sin2 y dy

(c) # q

0

1

(1 1 y2) (1 1 y2y2) dy (d) #

q

22 ye2y dy

cha9792x_ch22_633-654.indd Page 653 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 653 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

654 INTEGRATION OF EQUATIONS

(mg/m3). The following functional representations defi ne the tem- poral variations in fl ow and concentration:

Q(t) 5 9 1 5 cos2(0.4t)

c(t) 5 5e20.5t 1 2e0.15t

Determine the mass transported between t1 5 2 and t2 5 8 min with Romberg integration to a tolerance of 0.1%. 22.16 The depths of a river H are measured at equally spaced distances across a channel as tabulated below. The river’s cross- sectional area can be determined by integration as in

Ac 5 # x

0 H(x) dx

Use Romberg integration to perform the integration to a stopping criterion of 1%.

x, m 0 2 4 6 8 10 12 14 16

H, m 0 1.9 2 2 2.4 2.6 2.25 1.12 0

22.17 Recall that the velocity of the freefalling parachutist with linear drag can be computed analytically as

y(t) 5 gm

c (1 2 e2(cym)t)

where v(t) 5 velocity (m/s), t 5 time (s), g 5 9.81 m/s2, m 5 mass (kg), c 5 linear drag coeffi cient (kg/s). Use Romberg integration to compute how far the jumper travels during the fi rst 8 seconds of free fall given m 5 80 kg and c 5 10 kg/s. Compute the answer to es 5 1%. 22.18 Prove that Eq. (22.15) is equivalent to Boole’s rule.

(e) # q

0

1

22p e2x

2y2 dx

Note that (e) is the normal distribution (recall Fig. 22.10). 22.10 Develop a user-friendly computer program for the multiple- segment (a) trapezoidal and (b) Simpson’s 1y3 rule based on Fig. 22.1. Test it by integrating

# 1

0 x0.1(1.2 2 x) (1 2 e20(x21)) dx

Use the true value of 0.602298 to compute et for n 5 4. 22.11 Develop a user-friendly computer program for Romberg in- tegration based on Fig. 22.4. Test it by duplicating the results of Examples 22.3 and 22.4 and the function in Prob. 22.10. 22.12 Develop a user-friendly computer program for adaptive quadrature based on Fig. 22.5. Test it by solving Prob. 22.10. 22.13 Develop a user-friendly computer program for Gauss quadrature. Test it by duplicating the results of Examples 22.3 and 22.4 and the function in Prob. 22.10. 22.14 There is no closed form solution for the error function,

erf(a) 5 2

2p #

a

0 e2x

2 dx

Use the two-point Gauss quadrature approach to estimate erf(1.5). Note that the exact value is 0.966105. 22.15 The amount of mass transported via a pipe over a period of time can be computed as

M 5 # t2

t1

Q(t)c(t)dt

where M 5 mass (mg), t1 5 the initial time (min), t2 5 the fi nal time (min), Q(t) 5 fl ow rate (m3/min), and c(t) 5 concentration

cha9792x_ch22_633-654.indd Page 654 29/10/13 9:47 PM F-468 cha9792x_ch22_633-654.indd Page 654 29/10/13 9:47 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

655

23 C H A P T E R 23

Numerical Differentiation

We have already introduced the notion of numerical differentiation in Chap. 4. Recall that we employed Taylor series expansions to derive fi nite-divided-difference approxima- tions of derivatives. In Chap. 4, we developed forward, backward, and centered difference approximations of fi rst and higher derivatives. Recall that, at best, these estimates had errors that were O(h2)—that is, their errors were proportional to the square of the step size. This level of accuracy is due to the number of terms of the Taylor series that were retained during the derivation of these formulas. We will now illustrate how to develop more accurate formulas by retaining more terms.

23.1 HIGH-ACCURACY DIFFERENTIATION FORMULAS

As noted above, high-accuracy divided-difference formulas can be generated by includ- ing additional terms from the Taylor series expansion. For example, the forward Taylor series expansion can be written as [Eq. (4.21)]

f(xi11) 5 f(xi) 1 f ¿(xi)h 1 f –(xi)

2 h2 1 p (23.1)

which can be solved for

f ¿(xi) 5 f(xi11) 2 f(xi)

h 2

f –(xi) 2

h 1 O(h2) (23.2)

In Chap. 4, we truncated this result by excluding the second- and higher-derivative terms and were thus left with a fi nal result of

f ¿(xi) 5 f(xi11) 2 f(xi)

h 1 O(h) (23.3)

In contrast to this approach, we now retain the second-derivative term by substitut- ing the following approximation of the second derivative [recall Eq. (4.24)]

f –(xi) 5 f(xi12) 2 2 f(xi11) 1 f(xi)

h2 1 O(h) (23.4)

cha9792x_ch23_655-672.indd Page 655 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 655 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

656 NUMERICAL DIFFERENTIATION

into Eq. (23.2) to yield

f ¿(xi) 5 f(xi11) 2 f(xi)

h 2

f(xi12) 2 2 f(xi11) 1 f(xi)

2h2 h 1 O(h2)

or, by collecting terms,

f ¿(xi) 5 2f(xi12) 1 4 f(xi11) 2 3 f(xi)

2h 1 O(h2) (23.5)

Notice that inclusion of the second-derivative term has improved the accuracy to O(h2). Similar improved versions can be developed for the backward and centered for- mulas as well as for the approximations of the higher derivatives. The formulas are summarized in Figs. 23.1 through 23.3 along with all the results from Chap. 4. The following example illustrates the utility of these formulas for estimating derivatives.

FIGURE 23.1 Forward fi nite-divided-difference formulas: two versions are presented for each derivative. The latter version incorporates more terms of the Taylor series expansion and is, consequently, more accurate.

First Derivative Error

f ¿(xi) 5 f (xi11) 2 f (xi)

h O(h)

f ¿(xi) 5 2f (xi12) 1 4f (xi11) 2 3f (xi)

2h O(h2)

Second Derivative

f–(xi) 5 f (xi12) 2 2f (xi11) 1 f (xi)

h2 O(h)

f–(xi) 5 2f (xi13) 1 4f (xi12) 2 5f (xi11) 1 2f (xi)

h2 O(h2)

Third Derivative

f‡(xi) 5 f (xi13) 2 3f (xi12) 1 3f (xi11) 2 f (xi)

h3 O(h)

f‡(xi) 5 23f (xi14) 1 14f (xi13) 2 24f (xi12) 1 18f (xi11) 2 5f (xi)

2h3 O(h2)

Fourth Derivative

f––(xi) 5 f (xi14) 2 4f (xi13) 1 6f (xi12) 2 4f (xi11) 1 f (xi)

h4 O(h)

f––(xi) 5 22f (xi15) 1 11f (xi14) 2 24f (xi13) 1 26f (xi12) 2 14f (xi11) 1 3f (xi)

h4 O(h2)

cha9792x_ch23_655-672.indd Page 656 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 656 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.1 HIGH-ACCURACY DIFFERENTIATION FORMULAS 657

FIGURE 23.2 Backward fi nite-divided- difference formulas: two versions are presented for each derivative. The latter version incorporates more terms of the Taylor series expansion and is, consequently, more accurate.

First Derivative Error

f ¿(xi) 5 f (xi) 2 f (xi21)

h O(h)

f ¿(xi) 5 3f (xi) 2 4f (xi21) 1 f (xi22)

2h O(h2)

Second Derivative

f–(xi) 5 f (xi) 2 2f (xi21) 1 f (xi22)

h2 O(h)

f–(xi) 5 2f (xi) 2 5f (xi21) 1 4f (xi22) 2 f (xi23)

h2 O(h2)

Third Derivative

f‡(xi) 5 f (xi) 2 3f (xi21) 1 3f (xi22) 2 f (xi23)

h3 O(h)

f‡(xi) 5 5f (xi) 2 18f (xi21) 1 24f (xi22) 2 14f (xi23) 1 3f (xi24)

2h3 O(h2)

Fourth Derivative

f––(xi) 5 f (xi) 2 4f (xi21) 1 6f (xi22) 2 4f (xi23) 1 f (xi24)

h4 O(h)

f––(xi) 5 3f (xi) 2 14f (xi21) 1 26f (xi22) 2 24f  (xi23) 1 11f (xi24) 2 2f (xi25)

h4 O(h2)

FIGURE 23.3 Centered fi nite-divided- difference formulas: two versions are presented for each derivative. The latter version incorporates more terms of the Taylor series expansion and is, consequently, more accurate.

First Derivative Error

f ¿(xi) 5 f (xi11) 2 f (xi21)

2h O(h2)

f ¿(xi) 5 2f (xi12) 1 8f (xi11) 2 8f (xi21) 1 f (xi22)

12h O(h4)

Second Derivative

f–(xi) 5 f (xi11) 2 2f (xi) 1 f (xi21)

h2 O(h2)

f–(xi) 5 2f (xi12) 1 16f (xi11) 2 30f (xi) 1 16f (xi21) 2 f (xi22)

12h2 O(h4)

Third Derivative

f‡(xi) 5 f (xi12) 2 2f (xi11) 1 2f (xi21) 2 f (xi22)

2h3 O(h2)

f‡(xi) 5 2f (xi13) 1 8f (xi12) 2 13f (xi11) 1 13f (xi21) 2 8f (xi22) 1 f (xi23)

8h3 O(h4)

Fourth Derivative

f––(xi) 5 f (xi12) 2 4f (xi11) 1 6f (xi) 2 4f (xi21) 1 f (xi22)

h4 O(h2)

f––(xi) 5 2f (xi13) 1 12f (xi12) 2 39f (xi11) 1 56f (xi) 2 39f (xi21) 1 12f (xi22) 2 f (xi23)

6h4 O(h4)

cha9792x_ch23_655-672.indd Page 657 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 657 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

658 NUMERICAL DIFFERENTIATION

EXAMPLE 23.1 High-Accuracy Differentiation Formulas

Problem Statement. Recall that in Example 4.4 we estimated the derivative of

f(x) 5 20.1x4 2 0.15x3 2 0.5x2 2 0.25x 1 1.2

at x 5 0.5 using fi nite divided differences and a step size of h 5 0.25,

Forward Backward Centered O(h) O(h) O(h2)

Estimate 21.155 20.714 20.934 et (%) 226.5 21.7 22.4

where the errors were computed on the basis of the true value of 20.9125. Repeat this com- putation, but employ the high-accuracy formulas from Figs. 23.1 through 23.3.

Solution. The data needed for this example are

xi22 5 0 f(xi22) 5 1.2

xi21 5 0.25 f(xi21) 5 1.1035156

xi 5 0.5 f(xi) 5 0.925

xi11 5 0.75 f(xi11) 5 0.6363281

xi12 5 1 f(xi12) 5 0.2

The forward difference of accuracy O(h2) is computed as (Fig. 23.1)

f ¿(0.5) 5 20.2 1 4(0.6363281) 2 3(0.925)

2(0.25) 5 20.859375  et 5 5.82%

The backward difference of accuracy O(h2) is computed as (Fig. 23.2)

f ¿(0.5) 5 3(0.925) 2 4(1.1035156) 1 1.2

2(0.25) 5 20.878125  et 5 3.77%

The centered difference of accuracy O(h4) is computed as (Fig. 23.3)

f ¿(0.5) 5 20.2 1 8(0.6363281) 2 8(1.1035156) 1 1.2

12(0.25) 5 20.9125  et 5 0%

As expected, the errors for the forward and backward differences are considerably more accurate than the results from Example 4.4. However, surprisingly, the centered difference yields a perfect result. This is because the formulas based on the Taylor series are equivalent to passing polynomials through the data points.

23.2 RICHARDSON EXTRAPOLATION

To this point, we have seen that there are two ways to improve derivative estimates when employing fi nite divided differences: (1) decrease the step size or (2) use a higher-order formula that employs more points. A third approach, based on Richardson extrapolation, uses two derivative estimates to compute a third, more accurate approximation.

cha9792x_ch23_655-672.indd Page 658 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 658 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.2 RICHARDSON EXTRAPOLATION 659

Recall from Sec. 22.2.1 that Richardson extrapolation provided a means to obtain an improved integral estimate I by the formula [Eq. (22.4)]

I > I(h2) 1 1

(h1yh2) 2 2 1

[I(h2) 2 I(h1) ] (23.6)

where I(h1) and I(h2) are integral estimates using two step sizes h1 and h2. Because of its convenience when expressed as a computer algorithm, this formula is usually written for the case where h2 5 h1y2, as in

I > 4

3 I(h2) 2

1

3 I(h1) (23.7)

In a similar fashion, Eq. (23.7) can be written for derivatives as

D > 4

3 D(h2) 2

1

3 D(h1) (23.8)

For centered difference approximations with O(h2), the application of this formula will yield a new derivative estimate of O(h4).

EXAMPLE 23.2 Richardson Extrapolation

Problem Statement. Using the same function as in Example 23.1, estimate the fi rst derivative at x 5 0.5 employing step sizes of h1 5 0.5 and h2 5 0.25. Then use Eq. (23.8) to compute an improved estimate with Richardson extrapolation. Recall that the true value is 20.9125.

Solution. The fi rst-derivative estimates can be computed with centered differences as

D(0.5) 5 0.2 2 1.2

1 5 21.0  et 5 29.6%

and

D(0.25) 5 0.6363281 2 1.1035156

0.5 5 20.934375  et 5 22.4%

The improved estimate can be determined by applying Eq. (23.8) to give

D 5 4

3 (20.934375) 2

1

3 (21) 5 20.9125

which for the present case is a perfect result.

The previous example yielded a perfect result because the function being analyzed was a fourth-order polynomial. The perfect outcome was due to the fact that Richardson extrapolation is actually equivalent to fi tting a higher-order polynomial through these data and then evaluating the derivatives by centered divided differences. Thus, the pres- ent case matched the derivative of the fourth-order polynomial precisely. For most other functions, of course, this would not occur and our derivative estimate would be improved

cha9792x_ch23_655-672.indd Page 659 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 659 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

660 NUMERICAL DIFFERENTIATION

but not perfect. Consequently, as was the case for the application of Richardson ex- trapolation, the approach can be applied iteratively using a Romberg algorithm until the result falls below an acceptable error criterion.

23.3 DERIVATIVES OF UNEQUALLY SPACED DATA

The approaches discussed to this point are primarily designed to determine the derivative of a given function. For the fi nite-divided-difference approximations of Sec. 23.1, these data had to be evenly spaced. For the Richardson extrapolation technique of Sec. 23.2, these data had to be evenly spaced and generated for successively halved intervals. Such control of data spacing is usually available only in cases where we can use a function to generate a table of values. In contrast, empirically derived information—that is, data from experiments or fi eld studies—is often collected at unequal intervals. Such information cannot be analyzed with the techniques discussed to this point. One way to handle nonequispaced data is to fi t a second-order Lagrange interpolat- ing polynomial [recall Eq. (18.23)] to each set of three adjacent points. Remember that this polynomial does not require that the points be equispaced. The second-order poly- nomial can be differentiated analytically to give

f ¿(x) 5 f(xi21) 2x 2 xi 2 xi11

(xi21 2 xi)(xi21 2 xi11) 1 f(xi)

2x 2 xi21 2 xi11 (xi 2 xi21) (xi 2 xi11)

1 f(xi11) 2x 2 xi21 2 xi

(xi11 2 xi21) (xi11 2 xi) (23.9)

where x is the value at which you want to estimate the derivative. Although this equation is certainly more complicated than the fi rst-derivative approximations from Figs. 23.1 through 23.3, it has some important advantages. First, it can be used to estimate the derivative anywhere within the range prescribed by the three points. Second, the points themselves do not have to be equally spaced. Third, the derivative estimate is of the same accuracy as the centered difference [Eq. (4.22)]. In fact, for equispaced points, Eq. (23.9) evaluated at x 5 xi reduces to Eq. (4.22).

EXAMPLE 23.3 Differentiating Unequally Spaced Data

Problem Statement. As in Fig. 23.4, a temperature gradient can be measured down into the soil. The heat fl ux at the soil-air interface can be computed with Fourier’s law,

q(z 5 0) 5 2krC dT

dz ` z50

where q 5 heat fl ux (W/m2), k 5 coeffi cient of thermal diffusivity in soil (> 3.5 3 1027 m2/s), r 5 soil density (> 1800 kg/m3), and C 5 soil specifi c heat (> 840 J/(kg ? 8C)). Note that a positive value for fl ux means that heat is transferred from the air to the soil. Use numerical differentiation to evaluate the gradient at the soil-air interface and employ this estimate to determine the heat fl ux into the ground.

cha9792x_ch23_655-672.indd Page 660 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 660 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.4 DERIVATIVES AND INTEGRALS FOR DATA WITH ERRORS 661

Solution. Equation (23.9) can be used to calculate the derivative as

f ¿(x) 5 13.5 2(0) 2 1.25 2 3.75

(0 2 1.25)(0 2 3.75) 1 12

2(0) 2 0 2 3.75

(1.25 2 0)(1.25 2 3.75)

1 10 2(0) 2 0 2 1.25

(3.75 2 0)(3.75 2 1.25)

5 214.4 1 14.4 2 1.333333 5 21.333333°C/cm

which can be used to compute (note that 1 W 5 1 J/s),

q(z 5 0) 5 23.5 3 1027 m2

s a1800 kg

m3 b a840 J

kg ? °C b a2133.3333 °C

m b

5 70.56 W/m2

23.4 DERIVATIVES AND INTEGRALS FOR DATA WITH ERRORS

Aside from unequal spacing, another problem related to differentiating empirical data is that it usually includes measurement error. A shortcoming of numerical differentiation is that it tends to amplify errors in the data. Figure 23.5a shows smooth, error-free data that when numerically differentiated yield a smooth result (Fig. 23.5c). In contrast, Fig. 23.5b uses the same data, but with some points raised and some lowered slightly. This minor modifi cation is barely apparent from Fig. 23.5b. However, the resulting effect in Fig. 23.5d is signifi cant because the process of differentiation amplifi es errors. As might be expected, the primary approach for determining derivatives for imprecise data is to use least-squares regression to fi t a smooth, differentiable function to these data. In the absence of any other information, a lower-order polynomial regression might be a good fi rst choice. Obviously, if the true functional relationship between the dependent and inde- pendent variable is known, this relationship should form the basis for the least-squares fi t.

FIGURE 23.4 Temperature versus depth into the soil.

z, cm

T(�C)10Air

Soil

3.75

13.512

1.25

cha9792x_ch23_655-672.indd Page 661 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 661 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

662 NUMERICAL DIFFERENTIATION

23.4.1 Differentiation versus Integration of Uncertain Data

Just as curve-fi tting techniques like regression can be used to differentiate uncertain data, a similar process can be employed for integration. However, because of the difference in stability between differentiation and integration, this is rarely done. As depicted in Fig. 23.5, differentiation tends to be unstable—that is, it amplifi es errors. In contrast, the fact that integration is a summing process tends to make it very forgiving with regard to uncertain data. In essence, as points are summed to form an integral, random positive and negative errors tend to cancel out. In contrast, because differentiation is subtractive, random positive and negative errors tend to add.

23.5 PARTIAL DERIVATIVES

Partial derivatives along a single dimension are computed in the same fashion as ordinary derivatives. For example, suppose that we want to determine to partial derivatives for a two-dimensional function, f(x, y). For equally-spaced data, the partial fi rst derivatives can be approximated with centered differences,

0f 0x

5 f(x 1 ¢x, y) 2 f(x 2 ¢x, y)

2¢x (23.10)

0f 0y

5 f(x, y 1 ¢y) 2 f(x, y 2 ¢y)

2¢y (23.11)

FIGURE 23.5 Illustration of how small data errors are amplifi ed by numerical differentiation: (a) data with no error, (b) data modifi ed slightly, (c) the resulting numerical differentiation of curve (a), and (d) the resulting differentiation of curve (b) mani- festing increased variability. In contrast, the reverse operation of integration [moving from (d) to (b) by taking the area un- der (d)] tends to attenuate or smooth data errors.

y

t

(a)

D if

fe re

n ti

a te

y

t

(b)

D if

fe re

n ti

a te

t

(c) t

(d )

dy dt

dy dt

cha9792x_ch23_655-672.indd Page 662 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 662 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.6 NUMERICAL INTEGRATION/DIFFERENTIATION WITH SOFTWARE PACKAGES 663

All the other formulas and approaches discussed to this point can be applied to evaluate partial derivatives in a similar fashion. For higher-order derivatives, we might want to differentiate a function with respect to two or more different variables. The result is called a mixed partial derivative. For example, we might want to take the partial derivative of f(x, y) with respect to both independent variables

02 f 0x0y

5 0 0x

a 0f 0y b (23.12)

To develop a fi nite-difference approximation, we can fi rst form a difference in x of the partial derivatives in y,

02 f 0x0y

5

0f 0y

(x 1 ¢x, y) 2 0f 0y

(x 2 ¢x, y)

2¢x (23.13)

Then, we can use fi nite differences to evaluate each of the partials in y,

02 f 0x0y

5

f(x 1 ¢x, y 1 ¢y) 2 f(x 1 ¢x, y 2 ¢y) 2¢y

2 f(x 2 ¢x, y 1 ¢y) 2 f(x 2 ¢x, y 2 ¢y)

2¢y 2¢x

(23.14)

Collecting terms yields the fi nal result

02 f 0x0y

5 f(x 1 ¢x, y 1 ¢y) 2 f(x 1 ¢x, y 2 ¢y) 2 f(x 2 ¢x, y 1 ¢y) 1 f(x 2 ¢x, y 2 ¢y)

4¢x¢y (23.15)

23.6 NUMERICAL INTEGRATION/DIFFERENTIATION WITH SOFTWARE PACKAGES

Software packages have great capabilities for numerical integration and differentiation. In this section, we will give you a taste of some of the more useful ones.

23.6.1 MATLAB

MATLAB software has a variety of built-in functions that allow functions and data to be integrated and differentiated (Table 23.1). In this section, we will illustrate some of these capabilities. MATLAB can integrate both discrete data and functions. For example, trapz com- putes the integral of discrete values using the multiple-application trapezoidal rule. A simple representation of its syntax is

q = trapz(x, y)

where the two vectors, x and y, hold the independent and dependent variables, respec- tively, and q holds the resulting integral. It also has another function, cumtrapz, that computes the cumulative integral. For this case, the result is a vector whose elements q(k) hold the integral from x(1)to x(k).

S O F T W A R E

cha9792x_ch23_655-672.indd Page 663 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 663 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

664 NUMERICAL DIFFERENTIATION S O F T W A R E

When the integrand is available in functional form, quad generates the defi nite integral using adaptive quadrature. A simple representation of its syntax is

q = quad(fun, a, b)

where fun is the function to be integrated, and a and b are the integration limits.

EXAMPLE 23.4 Using Numerical Integration to Compute Distance from Velocity

Problem Statement. As described in Sec. PT6.1, integration can be used to compute the distance, y(t), of an object based on its velocity, v(t), as in,

y(t) 5 # t

0 y(t) dt (E23.4.1)

Recall from Sec. 1.1, that the velocity of a free-falling parachutist, subject to linear drag and with zero initial velocity, can be computed with

y(t) 5 gm

c (1 2 e2(cym)t) (E23.4.2)

If we substitute, Eq. (E23.4.2) into Eq. (E23.4.1), the result can be integrated analytically, with the initial condition, y(0) 5 0, to yield

y(t) 5 gm

c t 2

gm2

c2 (1 2 e2 (cym)t)

This result can be used to compute that a 70-kg parachutist with a drag coeffi cient of 12.5 kg/s will fall 799.73 m over a 20-s period. Use MATLAB functions to perform the same integration numerically. In addition, de- velop a plot of the analytical and computed distances along with velocity on the same graph.

TABLE 23.1 MATLAB functions to implement (a) integration and (b) differentiation.

Function Description

(a) Integration: cumtrapz Cumulative trapezoidal numerical integration dblquad Numerically evaluate double integral polyint Integrate polynomial analytically quad Numerically evaluate integral, adaptive Simpson quadrature quadgk Numerically evaluate integral, adaptive Gauss-Kronrod quadrature quadl Numerically evaluate integral, adaptive Lobatto quadrature quadv Vectorized quadrature trapz Trapezoidal numerical integration triplequad Numerically evaluate triple integral

(b) Differentiation: del2 Discrete Laplacian diff Differences and approximate derivatives gradient Numerical gradient polyder Polynomial derivative

cha9792x_ch23_655-672.indd Page 664 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 664 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.6 NUMERICAL INTEGRATION/DIFFERENTIATION WITH SOFTWARE PACKAGES 665

Solution. We can fi rst use Eq. (E23.4.2) to generate some unequally-spaced times and velocities. We can then round these velocities so that they are more like measured values; that is, they are not exact,

>> format short g >> t=[0 1 2 3 4.3 7 12 16]; >> g=9.81;m=70;c=12.5; >> v=round(g*m/c*(1-exp(-c/m*t)));

The total distance can then be computed as

>> y=trapz (t,v)

y = 789.6

Thus, after 20 seconds, the jumper has fallen 789.6 m, which is reasonably close to the exact, analytical solution of 799.73 m. If we desire the cumulative distance travelled at each time, cumtrapz can be em- ployed to compute,

>> yc=cumtrapz (t,v)

yc = 0 4.5 17 36.5 70.3 162.1 379.6 579.6 789.6

A graph of the numerical and analytical solutions along with both the exact and rounded velocities are generated with the following commands,

>> ta=linspace (t(1), t(length(t))); >> ya=g*m/c*ta-g*m^2/c^2*(1-exp(-c/m*ta)); >> plot (ta, ya, t, yc, 'o') >> title ('Distance versus time') >> xlabel ('t (s)'), ylabel ('x (m)') >> legend ('analytical', 'numerical')

As in Fig. 23.6, the numerical and analytical results match fairly well. Finally, the quad function can be used to evaluate the integral with adaptive quadrature

>> va=@(t) g*m/c*(1-exp(-c/m*t)); >> yq=quad(va,t(1),t(length(t)))

yq = 799.73

This result is identical to the analytical solution to within the 5 signifi cant digits displayed.

As listed in Table 23.1b, MATLAB has a number of built-in functions for evaluating derivatives including the diff and gradient functions. When it is passed a one- dimensional vector of length n, the diff function returns a vector of length n 2 1

cha9792x_ch23_655-672.indd Page 665 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 665 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

666 NUMERICAL DIFFERENTIATION

containing the differences between adjacent elements. These can then be employed to determine fi nite-difference approximations of fi rst-derivatives. The gradient function also returns differences. However, it does so in a manner that is more compatible with evaluating derivatives at the values themselves rather than in the intervals between values. A simple representation of its syntax is

fx = gradient(f)

where f 5 a one-dimensional vector of length n, and fx is a vector of length n contain- ing differences based on f. Just as with the diff function, the fi rst value returned is the difference between the fi rst and second value. However, for the intermediate values, a centered difference based on the adjacent values is returned,

diffi 5 fi11 2 fi21

2

The last value is then computed as the difference between the fi nal two values. Hence, the results correspond to using centered differences for all the intermediate values, with forward and backward differences at the ends. Note that the spacing between points is assumed to be one. If the vector represents equally-spaced data, the following version divides all the results by the interval and hence returns the actual values of the derivatives,

fx = gradient(f, h)

where h 5 the spacing between points.

EXAMPLE 23.5 Using diff and gradient for Differentiation

Problem Statement. Explore how the MATLAB’s diff and gradient functions can be employed to differentiate the function f(x) 5 0.2 1 25x 2 200x2 1 675x3 2 900x4 1

FIGURE 23.6 Plot of distance versus time. The line was computed with the analytical solution, whereas the points were determined numerically with the cumtrapz function.

S O F T W A R E

cha9792x_ch23_655-672.indd Page 666 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 666 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.6 NUMERICAL INTEGRATION/DIFFERENTIATION WITH SOFTWARE PACKAGES 667

400x5 from x 5 0 to 0.8. Compare your results with the exact solution: f9(x) 5 25 2 400x2 1 2025x2 2 3600x3 1 2000x4.

Solution. We can fi rst express f(x) as an anonymous function

>> f=@(x) 0.2+25*x-200*x.^2+675*x.^3-900*x.^4+400*x.^5;

We then generate a series of equally-spaced values of the independent and dependent variables,

>> x=0:0.1:0.8; >> y=f(x);

The diff function is to determine the differences between adjacent elements of each vector. For example,

>> format short g >> diff(x)

0.1000 0.1000 0.1000 0.1000 0.1000 0.1000 0.1000 0.1000

As expected, the result represents the differences between each pair of elements of x. To compute divided-difference approximations of the derivative, we merely perform a vector division of the y differences by the x differences by entering

>> d=diff(y)./diff(x)

10.89 -0.01 3.19 8.49 8.69 1.39 -11.01 -21.31

Note that because we are using equally-spaced values, after generating the x values, we could have simply performed the above computation concisely as

>> d=diff(f(x))/0.1;

The vector d now contains derivative estimates corresponding to the midpoint between adjacent elements. Therefore, in order to develop a plot of our results, we must fi rst generate a vector holding the x values for the midpoint of each interval

>> n=length(x); >> xm=(x(1:n-1)+x(2:n))./2;

We can compute values for the analytical derivative at a fi ner level of resolution to include on the plot for comparison.

>> xa=0:.01:.8; >> ya=25-400*xa+3*675*xa.^2-4*900*xa.^3+5*400*xa.^4;

A plot of the numerical and analytical estimates is then generated with

subplot (1, 2, 1), plot (xm, d, 'o', xa, ya) xlabel ('x'), ylabel ('y') legend ('numerical', 'analytical'),title ('(a) diff')

As displayed in Fig. 23.7a, the results of the numerical approximation compare favorably with the exact, analytical solution for this case.

cha9792x_ch23_655-672.indd Page 667 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 667 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

668 NUMERICAL DIFFERENTIATION

FIGURE 23.7 Comparison of the exact deriva- tive (line) with numerical esti- mates (circles) computed with MATLAB’s (a) diff, and (b) gradient functions.

We can also use the gradient function to determine the derivatives as

>> dy=gradient(y,0.1)

dy = 10.89 5.44 1.59 5.84 8.59 5.04 -4.81 -16.16 -21.31

As was done for the diff function, we can also display both the numerical and analytical estimates on a plot,

>> subplot(1,2,2), plot(x,dy,'o',xa,ya) >> xlabel('x') >> legend('numerical','analytical'),title('(b)gradient')

The results (Fig. 23.7b) are not as accurate as those obtained with the diff function (Fig. 23.7a). This is due to the fact that gradient employs intervals that are two times (0.2) as wide as for those used for diff (0.1).

Beyond one-dimensional vectors, the gradient function is particularly well-suited for determining the partial derivatives of matrices. For example, for a two-dimensional matrix, the function can be invoked as

[fx, fy] = gradient (f, h)

where f is a two-dimensional array, fx corresponds to the differences in the x (column) direction and fy corresponds to the differences in the y (row) direction, and h 5 the

S O F T W A R E

cha9792x_ch23_655-672.indd Page 668 09/12/13 9:12 AM F-468 cha9792x_ch23_655-672.indd Page 668 09/12/13 9:12 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

23.6 NUMERICAL INTEGRATION/DIFFERENTIATION WITH SOFTWARE PACKAGES 669

spacing between points. If h is omitted, the spacing between points in both dimensions is assumed to be one. In Sec. 31.4.2, we will illustrate how this capability can be used to visualize vector fi elds.

23.6.2 Mathcad

Mathcad has operators that perform numerical integration and differentiation. These operators employ and look like the same traditional mathematical symbols you have used since high school or your fi rst semester of college. The integration operator uses a sequence of trapezoidal rule evaluations of the integral and the Romberg algorithm. Iterations are performed until successive results vary by less than a tolerance. The derivative operator uses a similar method to compute derivatives between order 0 and 5. This operator creates a table of approximations based on divided- difference calculations of the derivative using various orders and step sizes. Extrapolation techniques are used to estimate values in a manner resembling Richardson’s method. Figure 23.8 shows a Mathcad example where f(x) is created using the defi nition symbol (:5), and then the integral is calculated over a range from x 5 0 to x 5 0.8. In this case, we used the simple polynomial we evaluated throughout Chap. 21. Note that the range as defi ned by the variables a and b is input with the defi nition symbol. Figure 23.9 shows a Mathcad example where a function f(x) is created with the defi nition symbol (:5) and then fi rst and third derivatives are calculated at a point where x 5 26. Note that the location of the point and the order of the derivative are input with the defi nition symbol.

FIGURE 23.8 Mathcad screen to determine the integral of a polynomial with Romberg integration.

cha9792x_ch23_655-672.indd Page 669 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 669 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

670 NUMERICAL DIFFERENTIATION

FIGURE 23.9 Mathcad screen to implement numerical differentiation.

PROBLEMS

23.1 Compute forward and backward difference approximations of O(h) and O(h2), and central difference approximations of O(h2) and O(h4) for the fi rst derivative of y 5 cos x at x 5 py4 using a value of h 5 py12. Estimate the true percent relative error et for each approximation. 23.2 Repeat Prob. 23.1, but for y 5 log x evaluated at x 5 25 with h 5 2. 23.3 Use centered difference approximations to estimate the fi rst and second derivatives of y 5 ex at x 5 2 for h 5 0.1. Employ both O(h2) and O(h4) formulas for your estimates. 23.4 Use Richardson extrapolation to estimate the fi rst deriva- tive of y 5 cos x at x 5 py4 using step sizes of h1 5 py3 and h2 5 py6. Employ centered differences of O(h

2) for the initial estimates. 23.5 Repeat Prob. 23.4, but for the fi rst derivative of ln x at x 5 5 using h1 5 2 and h2 5 1. 23.6 Employ Eq. (23.9) to determine the fi rst derivative of y 5 2x4 2 6x3 2 12x 2 8 at x 5 0 based on values at x0 5 20.5, x1 5 1, and x2 5 2. Compare this result with the true value and with an estimate obtained using a centered difference approximation based on h 5 1.

23.7 Prove that for equispaced data points, Eq. (23.9) reduces to Eq. (4.22) at x 5 xi. 23.8 Compute the fi rst-order central difference approximations of O(h4) for each of the following functions at the specifi ed location and for the specifi ed step size: (a) y 5 x3 1 4x 2 15 at x 5 0, h 5 0.25 (b) y 5 x2 cos x at x 5 0.4, h 5 0.1 (c) y 5 tan(xy3) at x 5 3, h 5 0.5 (d) y 5 sin(0.51x)yx at x 5 1, h 5 0.2 (e) y 5 ex 1 x at x 5 2, h 5 0.2 Compare your results with the analytical solutions. 23.9 The following data were collected for the distance traveled versus time for a rocket:

t, s 0 25 50 75 100 125

y, km 0 32 58 78 92 100

Use numerical differentiation to estimate the rocket’s velocity and acceleration at each time. 23.10 Develop a user-friendly program to apply a Romberg algo- rithm to estimate the derivative of a given function.

S O F T W A R E

cha9792x_ch23_655-672.indd Page 670 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 670 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 671

(a) Use MATLAB to integrate these data from x 5 21 to 1 and 22 to 2 with the trap function.

(b) Use MATLAB to estimate the infl ection points of these data. 23.16 Evaluate 0fy0x, 0fy0y, and 0fy(0x0y) for the following function at x 5 y 5 1 (a) analytically and (b) numerically Dx 5 Dy 5 0.0001,

f(x, y) 5 3xy 1 3x 2 x3 2 3y3

23.17 Evaluate the following integral with MATLAB,

# 2p

0

sin t

t dt

using both the quad and quadl functions. To learn more about quadl, type help quadl at the MATLAB prompt. 23.18 Use the diff command in MATLAB and compute the fi nite-difference approximation to the fi rst and second derivative at each x-value in the table below, excluding the two end points. Use fi nite-difference approximations that are second-order correct, O(Dx2).

x 0 1 2 3 4 5 6 7 8 9 10

y 1.4 2.1 3.3 4.8 6.8 6.6 8.6 7.5 8.9 10.9 10

23.19 The objective of this problem is to compare second-order accurate forward, backward, and centered fi nite-difference approx- imations of the fi rst derivative of a function to the actual value of the derivative. This will be done for

f(x) 5 e22x 2 x

(a) Use calculus to determine the correct value of the derivative at x 5 2.

(b) To evaluate the centered fi nite-difference approximations, start with x 5 0.5. Thus, for the fi rst evaluation, the x values for the centered difference approximation will be x 5 2 6 0.5 or x 5 1.5 and 2.5. Then, decrease in increments of 0.01 down to a minimum value of Dx 5 0.01.

(c) Repeat part (b) for the second-order forward and backward dif- ferences. (Note that these can be done at the same time that the centered difference is computed in the loop.)

(d) Plot the results of (b) and (c) versus x. Include the exact result on the plot for comparison.

23.20 Use a Taylor series expansion to derive a centered fi nite- difference approximation to the third derivative that is second-order

23.11 Develop a user-friendly program to obtain fi rst-derivative estimates for unequally spaced data. Test it with the following data:

x 1 1.5 1.6 2.5 3.5

f(x) 0.6767 0.3734 0.3261 0.08422 0.01596

where f(x) 5 5e22xx. Compare your results with the true derivatives. 23.12 The following data are provided for the velocity of an object as a function of time,

t, s 0 4 8 12 16 20 24 28 32 36

v, m/s 0 34.7 61.8 82.8 99.2 112.0121.9129.7135.7140.4

(a) Using the best numerical method available, how far does the object travel from t 5 0 to 28 s?

(b) Using the best numerical method available, what is the object’s acceleration at t 5 28 s?

(c) Using the best numerical method available, what is the object’s acceleration at t 5 0 s?

23.13 Recall that for the falling parachutist problem, the velocity is given by

y(t) 5 gm

c (1 2 e2(cym)t) (P23.13.1)

and the distance traveled can be obtained by

d(t) 5 gm

c #

t

0 (1 2 e2(cym)t) dt (P23.13.2)

Given g 5 9.81, m 5 70, and c 5 12, (a) Use MATLAB or Mathcad to integrate Eq. (P23.13.1) from

t 5 0 to 10. (b) Analytically integrate Eq. (P23.13.2) with the initial condition

that d 5 0 at t 5 0. Evaluate the result at t 5 10 to confi rm (a). (c) Use MATLAB or Mathcad to differentiate Eq. (P23.13.1) at

t 5 10. (d) Analytically differentiate Eq. (P23.13.1) at t 5 10 to confi rm (c). 23.14 The normal distribution is defi ned as

f(x) 5 1

22p e2x

2y2

(a) Use MATLAB or Mathcad to integrate this function from x 5 21 to 1 and from 22 to 2.

(b) Use MATLAB or Mathcad to determine the infl ection points of this function.

23.15 The following data were generated from the normal distribution:

x 22 21.5 21 20.5 0 0.5 1 1.5 2

f(x) 0.05399 0.12952 0.24197 0.35207 0.39894 0.35207 0.24197 0.12952 0.05399

cha9792x_ch23_655-672.indd Page 671 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 671 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

672 NUMERICAL DIFFERENTIATION

the shear stress t (N/m2) at the surface (y 5 0), using Newton’s viscosity law

t 5 m dy

dy

Assume a value of dynamic viscosity m 5 1.8 3 1025 N ? s/m2.

y, m 0 0.002 0.006 0.012 0.018 0.024

v, m/s 0 0.287 0.899 1.915 3.048 4.299

23.27 Chemical reactions often follow the model:

dc

dt 5 2kcn

where c 5 concentration, t 5 time, k 5 reaction rate, and n 5 reac- tion order. Given values of c and dcydt, k and n can be evaluated by a linear regression of the logarithm of this equation:

log a2dc dt b 5 log k 1 n log c

Use this approach along with the following data to estimate k and n:

t 10 20 30 40 50 60

c 3.52 2.48 1.75 1.23 0.87 0.61

23.28 The velocity profi le of a fl uid in a circular pipe can be repre- sented as

y 5 10 a1 2 r r0 b1yn

where v 5 velocity, r 5 radial distance measured out from the pipes centerline, r0 5 the pipe’s radius, and n 5 a parameter. Deter- mine the fl ow in the pipe if r0 5 0.75 and n 5 7 using (a) Romberg integration to a tolerance of 0.1%, (b) two-point Gauss-Legendre formula, and (c) the MATLAB quad function. Note that fl ow is equal to velocity times area. 23.29 The amount of mass transported via a pipe over a period of time can be computed as

M 5 # t2

t1 Q(t)c(t) dt

where M 5 mass (mg), t1 5 the initial time (min), t2 5 the fi nal time (min), Q(t) 5 fl ow rate (m3/min), and c(t) 5 concentration (mg/m3). The following functional representations defi ne the tem- poral variations in fl ow and concentration,

Q(t) 5 9 1 4cos2(0.4t)

c(t) 5 5e20.5t 1 2e0.15t

Determine the mass transported between t1 5 2 and t2 5 8 min with (a) Romberg integration to a tolerance of 0.1%, and (b) the MATLAB quad function.

accurate. To do this, you will have to use four different expansions for the points xi2 2, xi 2 1, xi 1 1, and xi 1 2. In each case, the expansion will be around the point xi. The interval Dx will be used in each case of i 2 1 and i 1 1, and 2Dx will be used in each case of i 2 2 and i 1 2. The four equations must then be combined in a way to elim- inate the fi rst and second derivatives. Carry enough terms along in each expansion to evaluate the fi rst term that will be truncated to determine the order of the approximation. 23.21 Use the following data to fi nd the velocity and acceleration at t 5 10 seconds:

Time, t, s 0 2 4 6 8 10 12 14 16

Position, x, m 0 0.7 1.8 3.4 5.1 6.3 7.3 8.0 8.4

Use second-order correct (a) centered fi nite-difference, (b) forward fi nite-difference, and (c) backward fi nite-difference methods. 23.22 A plane is being tracked by radar, and data are taken every second in polar coordinates u and r.

t, s 200 202 204 206 208 210

u, rad 0.75 0.72 0.70 0.68 0.67 0.66

r, m 5120 5370 5560 5800 6030 6240

At 206 s, use the centered fi nite difference (second-order correct) to fi nd the vector expressions for velocity y

S , and acceleration a

S . The

velocity and acceleration given in polar coordinates are:

y S

5 r # e S

r 1 ru # e S u and a

S 5 (r$ 2 r u

#  2) e S

r 1 (ru $

1 2r#u # ) e S u

23.23 Develop an Excel VBA macro program to read in adjacent columns of x and y values from a worksheet. Evaluate the deriva- tives at each point using Eq. 23.9, and display the results in a third column adjacent to the x and y values back on the spreadsheet. Test your program by applying it to evaluate the velocities for the time– position values from Prob. 23.21. 23.24 Use regression to estimate the acceleration at each time for the following data with second-, third-, and fourth-order polynomials. Plot the results.

t 1 2 3.25 4.5 6 7 8 8.5 9.3 10

v 10 12 11 14 17 16 12 14 14 10

23.25 You have to measure the fl ow rate of water through a small pipe. In order to do it, you place a bucket at the pipe’s outlet and measure the volume in the bucket as a function of time as tabulated below. Estimate the fl ow rate at t 5 7 s.

Time, s 0 1 5 8

Volume, cm3 0 1 8 16.4

23.26 The velocity y (m/s) of air fl owing past a fl at surface is mea- sured at several distances y (m) away from the surface. Determine

cha9792x_ch23_655-672.indd Page 672 05/11/13 5:46 PM F-468 cha9792x_ch23_655-672.indd Page 672 05/11/13 5:46 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24 C H A P T E R 24

673

Case Studies: Numerical Integration and Differentiation

The purpose of this chapter is to apply the methods of numerical integration and differen- tiation discussed in Part Six to practical engineering problems. Two situations are most frequently encountered. In the fi rst case, the function under study can be expressed in analytic form but is too complicated to be readily evaluated using the methods of calculus. Numerical methods are applied to situations of this type by using the analytic expression to generate a table of arguments and function values. In the second case, the function to be evaluated is inherently tabular in nature. This type of function usually represents a series of measurements, observations, or some other empirical information. Data for either case are directly compatible with several schemes discussed in this part of the book. Section 24.1, which deals with heat calculations from chemical engineering, involves equations. In this application, an analytic function is integrated numerically to determine the heat required to raise the temperature of a material. Sections 24.2 and 24.3 also involve functions that are available in equation form. Section 24.2, which is taken from civil engineering, uses numerical integration to deter- mine the total wind force acting on the mast of a racing sailboat. Section 24.3 determines the root-mean-square current for an electric circuit. This example is used to demonstrate the utility of Romberg integration and Gauss quadrature. Section 24.4 focuses on the analysis of tabular information to determine the work required to move a block. Although this application has a direct connection with me- chanical engineering, it is germane to all other areas of engineering. Among other things, we use this example to illustrate the integration of unequally spaced data.

24.1 INTEGRATION TO DETERMINE THE TOTAL QUANTITY OF HEAT (CHEMICAL/BIO ENGINEERING)

Background. Heat calculations are employed routinely in chemical and bio engineer- ing as well as in many other fi elds of engineering. This application provides a simple but useful example of such computations. One problem that is often encountered is the determination of the quantity of heat required to raise the temperature of a material. The characteristic that is needed to carry out this computation is the heat capacity c. This parameter represents the quantity of

cha9792x_ch24_673-693.indd Page 673 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 673 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

674 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

heat required to raise a unit mass by a unit temperature. If c is constant over the range of temperatures being examined, the required heat DH (in calories) can be calculated by

¢H 5 mc ¢T (24.1)

where c has units of cal/(g ? 8C), m 5 mass (g), and DT 5 change in temperature (8C). For example, the amount of heat required to raise 20 g of water from 5 to 108C is equal to

¢H 5 20(1)(10 2 5) 5 100 cal

where the heat capacity of water is approximately 1 cal/(g ? 8C). Such a computation is adequate when DT is small. However, for large ranges of temperature, the heat capacity is not constant and, in fact, varies as a function of temperature. For example, the heat capac- ity of a material could increase with temperature according to a relationship such as

c(T) 5 0.132 1 1.56 3 1024T 1 2.64 3 1027T 2 (24.2)

In this instance you are asked to compute the heat required to raise 1000 g of this material from 2100 to 2008C.

Solution. Equation (PT6.4) provides a way to calculate the average value of c(T):

c(T) 5

# T2

T1 c(T) dT

T2 2 T1 (24.3)

which can be substituted into Eq. (24.1) to yield

¢H 5 m # T2

T1 c(T) dT (24.4)

where DT 5 T2 2 T1. Now because, for the present case, c(T) is a simple quadratic, DH can be determined analytically. Equation (24.2) is substituted into Eq. (24.4) and the result integrated to yield an exact value of DH 5 42,732 cal. It is useful and instructive to compare this result with the numerical methods developed in Chap. 21. To accomplish this, it is necessary to generate a table of values of c for various values of T:

T, 8C c, cal/(g ? 8C)

2100 0.11904 250 0.12486 0 0.13200 50 0.14046 100 0.15024 150 0.16134 200 0.17376

These points can be used in conjunction with a six-segment Simpson’s 1y3 rule to com- pute an integral estimate of 42,732. This result can be substituted into Eq. (24.4) to yield a value of DH 5 42,732 cal, which agrees exactly with the analytical solution. This exact agreement would occur no matter how many segments were used. This is to be expected because c is a quadratic function and Simpson’s rule is exact for polynomials of the third order or less (see Sec. 21.2.1).

cha9792x_ch24_673-693.indd Page 674 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 674 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24.2 EFFECTIVE FORCE ON THE MAST OF A RACING SAILBOAT 675

The results using the trapezoidal rule are listed in Table 24.1. It is seen that the trapezoidal rule is also capable of estimating the total heat very accurately. However, a small step (, 108C) is required for fi ve-place accuracy. The same calculation can also be implemented with software. For example, MATLAB software yields

>> m=1000; >> DH=m*quad(@(T) 0.132+1.56e-4*T+2.64e-7*T.^2,-100,200)

DH = 42732

24.2 EFFECTIVE FORCE ON THE MAST OF A RACING SAILBOAT (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. A cross section of a racing sailboat is shown in Fig. 24.1a. Wind forces ( f ) exerted per foot of mast from the sails vary as a function of distance above the deck

TABLE 24.1 Results using the trapezoidal rule with various step sizes.

Step Size, 8C DH Et (%)

300 96,048 125 150 43,029 0.7 100 42,864 0.3 50 42,765 0.07 25 42,740 0.018 10 42,733.3 ,0.01 5 42,732.3 ,0.01 1 42,732.01 ,0.01 0.05 42,732.00003 ,0.01

Wind

z = 30 ft

z = 0

Mast support cables

Mast

T

3 ft

(b)

(a)

FIGURE 24.1 (a) Cross section of a racing sailboat. (b) Wind forces f exerted per foot of mast as a function of distance z above the deck of the boat.

cha9792x_ch24_673-693.indd Page 675 06/11/13 5:50 PM F-468 cha9792x_ch24_673-693.indd Page 675 06/11/13 5:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

676 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

of the boat (z), as in Fig. 24.1b. Calculate the tensile force T in the left mast support cable, assuming that the right support cable is completely slack and the mast joins the deck in a manner that transmits horizontal or vertical forces but no moments. Assume that the mast remains vertical.

Solution. To proceed with this problem, it is required that the distributed force f be converted to an equivalent total force F and that its effective location above the deck d be calculated (Fig. 24.2). This computation is complicated by the fact that the force exerted per foot of mast varies with the distance above the deck. The total force exerted on the mast can be expressed as the integral of a continuous function:

F 5 # 30

0 200 a z

5 1 z b e22zy30 dz (24.5)

This nonlinear integral is diffi cult to evaluate analytically. Therefore, it is convenient to employ numerical approaches such as Simpson’s rule and the trapezoidal rule for this problem. This is accomplished by calculating f(z) for various values of z and then using Eq. (21.10) or (21.18). For example, Table 24.2 has values of f(z) for a step size of 3 ft that provide data for Simpson’s 1y3 rule or the trapezoidal rule. Results for several step sizes are given in Table 24.3. It is observed that both methods give a value of F 5 1480.6 lb as the step size becomes small. In this case, step sizes of 0.05 ft for the trapezoidal rule and 0.5 ft for Simpson’s rule provide good results.

TABLE 24.3 Values of F computed on the basis of various versions of the trapezoidal rule and Simpson’s 1/3 rule.

Technique Step Size, ft Segments F, lb

Trapezoidal rule 15 2 1001.7 10 3 1222.3 6 5 1372.3 3 10 1450.8 1 30 1477.1 0.5 60 1479.7 0.25 120 1480.3 0.1 300 1480.5 0.05 600 1480.6 Simpson’s 1/3 rule 15 2 1219.6 5 6 1462.9 3 10 1476.9 1 30 1480.5 0.5 60 1480.6

TABLE 24.2 Values of f(z) for a step size of 3 ft that provide data for the trapezoidal rule and Simpson’s 1/3 rule.

z, ft 0 3 6 9 12 15 18 21 24 27 30

f(z), lb/ft 0 61.40 73.13 70.56 63.43 55.18 47.14 39.83 33.42 27.89 23.20

FIGURE 24.2 Free-body diagram of the forces exerted on the mast of a sailboat.

0 3 ft

d = 13.05 ft

V

T

H

F = 1480.6 lb

� = tan–1 (3/30) = 0.0996687

cha9792x_ch24_673-693.indd Page 676 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 676 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24.3 ROOT-MEAN-SQUARE CURRENT BY NUMERICAL INTEGRATION 677

The effective line of action of d (Fig. 24.2) can be calculated by evaluation of the integral

d 5 #

30

0 z f(z) dz

# 30

0 f(z) dz

(24.6)

or

d 5 #

30

0 200z[zy(5 1 z)]e22zy30 dz

1480.6 (24.7)

This integral can be evaluated using methods similar to the above. For example, Simpson’s 1y3 rule with a step size of 0.5 gives d 5 19,326.9y1480.6 5 13.05 ft. With F and d known from numerical methods, a free-body diagram is used to develop force and moment balance equations. This free-body diagram is shown in Fig. 24.2. Sum- ming forces in the horizontal and vertical direction and taking moments about point 0 gives

gFH 5 0 5 F 2 T sin u 2 H (24.8) gFV 5 0 5 V 2 T cos u (24.9) gM0 5 0 5 3V 2 Fd (24.10)

where T 5 the tension in the cable and H and V 5 the unknown reactions on the mast transmitted by the deck. The direction, as well as the magnitude, of H and V is unknown. Equation (24.10) can be solved directly for V because F and d are known.

V 5 Fd

3 5

(1480.6)(13.05)

3 5 6440.6 lb

Therefore, from Eq. (24.9),

T 5 V

cos u 5

6440.6

0.995 5 6473 lb

and from Eq. (24.8),

H 5 F 2 T sin u 5 1480.6 2 (6473)(0.0995) 5 836.54 lb

These forces now enable you to proceed with other aspects of the structural design of the boat such as the cables and the deck support system for the mast. This problem illustrates nicely two uses of numerical integration that may be encountered during the engineering design of structures. It is seen that both the trapezoidal rule and Simpson’s 1y3 rule are easy to apply and are practical problem-solving tools. Simpson’s 1y3 rule is more accurate than the trapezoidal rule for the same step size and thus may often be preferred.

24.3 ROOT-MEAN-SQUARE CURRENT BY NUMERICAL INTEGRATION (ELECTRICAL ENGINEERING)

Background. The average value of an oscillating electric current over one period may be zero. For example, suppose that the current is described by a simple sinusoid: i(t) 5 sin(2ptyT), where T is the period. The average value of this function can be determined

cha9792x_ch24_673-693.indd Page 677 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 677 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

678 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

by the following equation:

i 5 #

T

0 sin a2pt

T b dt

T 2 0 5

2cos (2p) 1 cos 0

T 5 0

Despite the fact that the net result is zero, such current is capable of performing work and generating heat. Therefore, electrical engineers often characterize such current by

IRMS 5 B

1

T # T

0 i2(t) dt (24.11)

where i(t) 5 the instantaneous current. Calculate the RMS or root-mean-square current of the waveform shown in Fig. 24.3 using the trapezoidal rule, Simpson’s ly3 rule, Romberg integration, and Gauss quadrature for T 5 1 s.

Solution. Integral estimates for various applications of the trapezoidal rule and Simpson’s 1y3 rule are listed in Table 24.4. Notice that Simpson’s rule is more accurate than the trapezoidal rule. The exact value for the integral is 15.41261. This result is obtained using a 128- segment trapezoidal rule or a 32-segment Simpson’s rule. The same estimate is also determined using Romberg integration (Fig. 24.4). In addition, Gauss quadrature can be used to make the same estimate. The determi- nation of the root-mean-square current involves the evaluation of the integral (T 5 1)

I 5 # 1y2

0 (10e2t sin 2pt)2 dt (24.12)

i

0 T/2T/4 t

For 0 � t � T/2, i(t) = 10e– t /T sin �2� � For T/2 � t � T, i(t) = 0

t T

FIGURE 24.3 A periodically varying electric current.

cha9792x_ch24_673-693.indd Page 678 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 678 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24.3 ROOT-MEAN-SQUARE CURRENT BY NUMERICAL INTEGRATION 679

TABLE 24.4 Values for the integral calculated using various numerical schemes. The percent relative error et is based on a true value of 15.41261.

Technique Segments Integral Et (%)

Trapezoidal rule 1 0.0 100 2 15.16327 1.62 4 15.40143 0.0725 8 15.41196 4.21 3 1023

16 15.41257 2.59 3 1024

32 15.41261 1.62 3 1025

64 15.41261 1.30 3 1026

128 15.41261 0

Simpson’s 1/3 rule 2 20.21769 231.2 4 15.48082 20.443 8 15.41547 20.0186 16 15.41277 1.06 3 1023

32 15.41261 0

First, a change in variable is performed by applying Eqs. (22.29) and (22.30) to yield

t 5 1

4 1

1

4 td   dt 5

1

4 dtd

These relationships can be substituted into Eq. (24.12) to yield

I 5 # 1

21 c10e2[1y41 (1y4)td] sin 2p a1

4 1

1

4 tdbd

2

1

4 dt (24.13)

For the two-point Gauss-Legendre formula, this function is evaluated at td 5 21y13 and 1y13, with the results being 7.684096 and 4.313728, respectively. These values can be substituted into Eq. (22.23) to yield an integral estimate of 11.99782, which represents an error of et 5 22.1%. The three-point formula is (Table 22.1)

I 5 0.5555556(1.237449) 1 0.8888889(15.16327) 1 0.5555556(2.684915)

5 15.65755   0 et 0 5 1.6 The results of using the higher-point formulas are summarized in Table 24.5.

O(h2) O(h4) O(h6) O(h8) O(h10) O(h12)

0 20.21769 15.16503 15.41502 15.41261 15.41261 15.16327 15.48082 15.41111 15.41262 15.41261 15.40143 15.41547 15.41225 15.41261 15.41196 15.41277 15.41261 15.41257 15.41262 15.41261

FIGURE 24.4 Result of using Romberg integration to estimate the RMS current.

cha9792x_ch24_673-693.indd Page 679 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 679 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

680 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

TABLE 24.5 Results of using various-point Gauss quadrature formulas to approximate the integral.

Points Estimate Et (%)

2 11.9978243 22.1 3 15.6575502 21.59 4 15.4058023 4.42 3 1022

5 15.4126391 22.01 3 1024

6 15.4126109 21.82 3 1025

The integral estimate of 15.41261 can be substituted into Eq. (24.12) to compute an IRMS of 3.925890 A. This result could then be employed to guide other aspects of the design and operation of the circuit.

24.4 NUMERICAL INTEGRATION TO COMPUTE WORK (MECHANICAL/AEROSPACE ENGINEERING)

Background. Many engineering problems involve the calculation of work. The general formula is

Work 5 force 3 distance

When you were introduced to this concept in high school physics, simple applications were presented using forces that remained constant throughout the displacement. For example, if a force of 10 lb was used to pull a block a distance of 15 ft, the work would be calculated as 150 ft ? lb. Although such a simple computation is useful for introducing the concept, realistic problem settings are usually more complex. For example, suppose that the force varies during the course of the calculation. In such cases, the work equation is reexpressed as

W 5 # xn

x0 F(x) dx (24.14)

where W 5 work (ft ? lb), x0 and xn 5 the initial and fi nal positions, respectively, and F(x) a force that varies as a function of position. If F(x) is easy to integrate, Eq. (24.14) can be evaluated analytically. However, in a realistic problem setting, the force might not be expressed in such a manner. In fact, when analyzing measured data, the force might be available only in tabular form. For such cases, numerical integration is the only viable option for the evaluation. Further complexity is introduced if the angle between the force and the direction of movement also varies as a function of position (Fig. 24.5). The work equation can be modifi ed further to account for this effect, as in

W 5 # xn

x0 F(x)cos [u(x)] dx (24.15)

Again, if F(x) and u(x) are simple functions, Eq. (24.15) might be solved analytically. How- ever, as in Fig. 24.5, it is more likely that the functional relationship is complicated. For this situation, numerical methods provide the only alternative for determining the integral.

cha9792x_ch24_673-693.indd Page 680 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 680 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24.4 NUMERICAL INTEGRATION TO COMPUTE WORK 681

Suppose that you have to perform the computation for the situation depicted in Fig. 24.5. Although the fi gure shows the continuous values for F(x) and u(x), assume that, because of experimental constraints, you are provided with only discrete measurements at x 5 5-ft intervals (Table 24.6). Use single- and multiple-application versions of the trapezoidal rule and Simpson’s 1y3 and 3y8 rules to compute work for this data.

FIGURE 24.5 The case of a variable force acting on a block. For this case, the angle, as well as the magni- tude, of the force varies.

F(x)

x0

0 0

30 x, ft

10

F (x

), lb

1

0 0

30 x, ft

� (x

), r

ad

F(x)

xn

TABLE 24.6 Data for force F(x) and angle u(x) as a function of position x.

x, ft F(x), lb U, rad F(x) cos U

0 0.0 0.50 0.0000 5 9.0 1.40 1.5297 10 13.0 0.75 9.5120 15 14.0 0.90 8.7025 20 10.5 1.30 2.8087 25 12.0 1.48 1.0881 30 5.0 1.50 0.3537

cha9792x_ch24_673-693.indd Page 681 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 681 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

682 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

Solution. The results of the analysis are summarized in Table 24.7. A percent relative error et was computed in reference to a true value of the integral of 129.52 that was estimated on the basis of values taken from Fig. 24.5 at 1-ft intervals. The results are interesting because the most accurate outcome occurs for the simple two-segment trapezoidal rule. More refi ned estimates using more segments, as well as Simpson’s rules, yield less accurate results. The reason for this apparently counterintuitive result is that the coarse spacing of the points is not adequate to capture the variations of the forces and angles. This is particularly evident in Fig. 24.6, where we have plotted the continuous curve for the product of F(x) and cos [u(x)]. Notice how the use of seven points to characterize the continuously vary- ing function misses the two peaks at x 5 2.5 and 12.5 ft. The omission of these two points effectively limits the accuracy of the numerical integration estimates in Table 24.7. The fact that the two-segment trapezoidal rule yields the most accurate result is due to the chance positioning of the points for this particular problem (Fig. 24.7). The conclusion to be drawn from Fig. 24.6 is that an adequate number of measure- ments must be made to accurately compute integrals. For the present case, if data were

TABLE 24.7 Estimates of work calculated using the trapezoidal rule and Simpson’s rules. The percent relative error et as computed in reference to a true value of the integral (129.52 ft ? lb) that was estimated on the basis of values at 1-ft intervals.

Technique Segments Work Et, %

Trapezoidal 1 5.31 95.9 2 133.19 2.84 3 124.98 3.51 6 119.09 8.05 Simpson’s 1/3 rule 2 175.82 235.75 6 117.13 9.57 Simpson’s 3/8 rule 3 139.93 28.04

x, ft 0 30

F (x

) co

s [�

( x)

]

Work

FIGURE 24.6 A continuous plot of F(x) cos [u(x)] versus position with the seven discrete points used to develop the numerical integration estimates in Table 24.7. Notice how the use of seven points to characterize this continuously varying function misses two peaks at x 5 2.5 and 12.5 ft.

cha9792x_ch24_673-693.indd Page 682 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 682 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

24.4 NUMERICAL INTEGRATION TO COMPUTE WORK 683

available at F(2.5) cos [u(2.5)] 5 4.3500 and F(12.5) cos [u(12.5)] 5 11.3600, we could determine an integral estimate using the algorithm for unequally spaced data described previously in Sec. 21.3. Figure 24.8 illustrates the unequal segmentation for this case. Including the two additional points yields an improved integral estimate of 126.9 (et 5 2.02%). Thus, the inclusion of the additional data would incorporate the peaks that were missed previously and, as a consequence, lead to better results.

FIGURE 24.7 Graphical depiction of why the two-segment trapezoidal rule yields a good estimate of the integral for this particular case. By chance, the use of two trapezoids happens to lead to an even balance between positive and negative errors.

x, ft

0 0

10

30 F

(x )

co s

[� (

x) ]

Overestimates

Underestimates

FIGURE 24.8 The unequal segmentation scheme that results from the inclusion of two additional points at x 5 2.5 and 12.5 in the data in Table 24.6. The numerical integration formulas applied to each set of segments are shown.

x 0

S im

p so

n ’s

1 /3

T ra

p ez

o id

al

S im

p so

n ’s

1 /3

S im

p so

n ’s

3 /8

0 30

10

F (x

) co

s [�

( x)

]

cha9792x_ch24_673-693.indd Page 683 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 683 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

684 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

PROBLEMS

Chemical/Bio Engineering 24.1 Perform the same computation as Sec. 24.1, but compute the amount of heat required to raise the temperature of 1200 g of the material from 2150 to 1008C. Use Simpson’s rule for your compu- tation, with values of T at 508C increments. 24.2 Repeat Prob. 24.1, but use Romberg integration to es 5 0.01%. 24.3 Repeat Prob. 24.1, but use a two- and a three-point Gauss- Legendre formula. Interpret your results. 24.4 Integration provides a means to compute how much mass enters or leaves a reactor over a specifi ed time period, as in

M 5 # t2

t1 Qc dt

where t1 and t2 5 the initial and fi nal times, respectively. This for- mula makes intuitive sense if you recall the analogy between inte- gration and summation. Thus, the integral represents the summation of the product of fl ow times concentration to give the total mass entering or leaving from t1 to t2. If the fl ow rate is constant, Q can be moved outside the integral:

M 5 Q # t2

t1 c dt (P24.4.1)

Use numerical integration to evaluate this equation for the data listed below. Note that Q 5 4 m3/min.

t, min 0 10 20 30 35 40 45 50

c, mg/m3 10 35 55 52 40 37 32 34

24.5 Use numerical integration to compute how much mass leaves a reactor based on the following measurements.

t, min 0 10 20 30 35 40 45 50

Q, m3/min 4 4.8 5.2 5.0 4.6 4.3 4.3 5.0

c, mg/m3 10 35 55 52 40 37 32 34

24.6 Fick’s fi rst diffusion law states that

Mass flux 5 2D dc

dx (P24.6.1)

where mass fl ux 5 the quantity of mass that passes across a unit area per unit time (g/cm2/s), D 5 a diffusion coeffi cient (cm2/s), c 5 concentration, and x 5 distance (cm). An environmental engi- neer measures the following concentration of a pollutant in the

sediments underlying a lake (x 5 0 at the sediment-water interface and increases downward):

x, cm 0 1 3

c, 1026 g/cm3 0.06 0.32 0.6

Use the best numerical differentiation technique available to esti- mate the derivative at x 5 0. Employ this estimate in conjunction with Eq. (P24.6.1) to compute the mass fl ux of pollutant out of the sediments and into the overlying waters (D 5 1.52 3 1026 cm2/s). For a lake with 3.6 3 106 m2 of sediments, how much pollutant would be transported into the lake over a year’s time? 24.7 The following data were collected when a large oil tanker was loading:

t, min 0 10 20 30 45 60 75

V, 106 barrels 0.4 0.7 0.77 0.88 1.05 1.17 1.35

Calculate the fl ow rate Q (that is, dVydt) for each time to the order of h2. 24.8 You are interested in measuring the fl uid velocity in a narrow rectangular open channel carrying petroleum waste between loca- tions in an oil refi nery. You know that, because of bottom friction, the velocity varies with depth in the channel. If your technician has time to perform only two velocity measurements, at what depths would you take them to obtain the best estimate of the average ve- locity? State your recommendation in terms of the percent of total depth d measured from the fl uid surface. For example, measuring at the top would be 0%d, whereas at the very bottom would be 100%d. 24.9 Soft tissue follows an exponential deformation behavior in uniaxial tension while it is in the physiologic or normal range of elongation. This can be expressed as

s 5 Eo a

(eae 2 1)

where s 5 stress, e 5 strain, and Eo and a are material constants that are determined experimentally. To evaluate the two material constants, the above equation is differentiated with respect to e, which is a fundamental relationship for soft tissue

ds

de 5 Eo 1 as

To evaluate Eo and a, stress-strain data are used to plot dsyde versus s and the slope and intercept of this plot are the two material con- stants, respectively. The table contains stress-strain data for heart chordae tendineae (small tendons use to hold heart valves closed during contraction of the heart muscle). This is data from loading the tissue; different curves are produced on unloading.

cha9792x_ch24_673-693.indd Page 684 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 684 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 685

24.10 The standard technique for determining cardiac output is the indicator dilution method developed by Hamilton. One end of a small catheter is inserted into the radial artery and the other end is connected to a densitometer, which can automatically record the con- centration of the dye in the blood. A known amount of dye, 5.6 mg, is injected rapidly, and the following data are obtained:

Time, Concentration, Time, Concentration, s mg/L s mg/L

5 0 21 2.3 7 0.1 23 1.1 9 0.11 25 0.9 11 0.4 27 1.75 13 4.1 29 2.06 15 9.1 31 2.25 17 8 33 2.32 19 4.2 35 2.43

Plotting the above data results in the dye dilution curve in Fig. P24.10a. The concentration reaches a maximum value at about 15 seconds and then falls off, followed by a rise due to the recircula- tion of dye. The curve is replotted on a semilog graph in Fig. P24.10b. Notice that a straight line approximates the descending limb of the

(a) Calculate the derivative dsyde using fi nite differences that are second-order accurate. Plot the data and eliminate the data points near the zero points that appear not to follow the straight- line relationship. The error in these data comes from the inabil- ity of the instrumentation to read the small values in this region. Perform a regression analysis of the remaining data points to determine the values of Eo and a. Plot the stress versus strain data points along with the analytic curve expressed by the fi rst equation. This will indicate how well the analytic curve matches these data.

(b) Often the previous analysis does not work well because the value of Eo is diffi cult to evaluate. To solve this problem, Eo is not used. A data point is selected (s, e) that is in the middle of the range used for the regression analysis. These values are substituted into the fi rst equation, and a value for Eoya is deter- mined and substituted into the fi rst equation:

s 5 a s eae 2 1

b (eae 2 1) Using this approach, experimental data that are well defi ned will produce a good match of the data points and the analytic curve. Use this new relationship and again plot the stress versus the strain data points and the new analytic curve.

s 3 103 N/m2 87.8 96.6 176 263 350 569 833 1227 1623 2105 2677 3378 4257

e 3 1023 m/m 153 198 270 320 355 410 460 512 562 614 664 716 766

2

4

6

8

10 20 30 0

10

0 40 Time after injection (s)

(a)

c

0.1

10

0 40

1

10 20 30 Time after injection (s)

log(c)

(b)

FIGURE P24.10

cha9792x_ch24_673-693.indd Page 685 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 685 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

686 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

collected the following data on the mass fl ux of insulin being deliv- ered through the patch (and skin) as a function of time:

Flux, mg/cm2/h Time, h Flux, mg/cm2/h Time, h

15 0 8 5 14 1 5 10 12 2 2.5 15 11 3 2 20 9 4 1 24

Remember that mass fl ux is fl ow rate through an area or (1yA) dmydt. Provide your best possible estimate for the amount of drug delivered through the skin in 24 hours using a 12 cm2 patch. 24.13 Videoangiography is used to measure blood fl ow and deter- mine the status of circulatory function. In order to quantify the videoangiograms, blood vessel diameter and blood velocity are needed such that total blood fl ow is determined. Below is the den- sitometric profi le taken from a videoangiogram of a blood vessel. One way to determine consistently where the edge of the blood vessel is from the angiogram is to determine where the fi rst deriva- tive of the profi le is an extreme value. Using the data provided, fi nd

dilution curve. In order to separate out the recirculation effect, ana- lysts extend the straight-line portion. The cardiac output can then be calculated from

C 5 M

A 3 60 s/min

where C 5 cardiac output [L/min], M 5 amount of injected dye (mg), and A 5 area under the curve with the linear correction. Cal- culate the cardiac output of this patient using the trapezoidal rule with a step size of 2 s. 24.11 Glaucoma is the second leading cause of vision loss world- wide. High intraocular pressure (pressure inside the eye) almost always accompanies vision loss. It is postulated that the high pres- sure damages a subset of cells in the eye that are responsible for vision. One investigator theorizes that the relationship between vision loss and pressure can be described as

VL 5 A exp ak # t

25 (P 2 13) dtb

where VL is percent vision loss, P is intraocular pressure (mm Hg), t is time (years), and k and A are constants. Using the data below from three patients, estimate the constants k and A.

Patient A B C

Age at diagnosis 65 43 80 VL 60 40 30

Age, years P, mm Hg Age, years P, mm Hg Age, years P, mm Hg

25 13 25 11 25 13 40 15 40 30 40 14 50 22 41 32 50 15 60 23 42 33 60 17 65 24 43 35 80 19

Distance Density Distance Density Distance Density Distance Density

0 26.013 28 38.273 56 39.124 84 37.331 4 26.955 32 39.103 60 38.813 88 35.980 8 26.351 36 39.025 64 38.925 92 31.936 12 28.343 40 39.432 68 38.804 96 28.843 16 31.100 44 39.163 72 38.806 100 26.309 20 34.667 48 38.920 76 38.666 104 26.146 24 37.251 52 38.631 80 38.658

24.12 One of your colleagues has designed a new transdermal patch to deliver insulin through the skin to diabetic patients in a controlled way, eliminating the need for painful injections. She has

the boundaries of the blood vessel and estimate the blood vessel diameter. Use both O(h2) and O(h4) centered difference formulas and compare the results.

cha9792x_ch24_673-693.indd Page 686 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 686 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 687

where U 5 water velocity (m/s). Use these relationships and a numerical method to determine Ac and Q for the following data:

y, m 0 2 4 5 6 9

H, m 0.5 1.3 1.25 1.7 1 0.25

U, m/s 0.03 0.06 0.05 0.12 0.11 0.02

24.19 The following relationships can be used to analyze uniform beams subject to distributed loads,

dy

dx 5 u(x) 

du

dx 5

M(x)

EI  

dM

dx 5 V(x) 

dV

dx 5 2w(x)

where x 5 distance along beam (m), y 5 defl ection (m), u(x) 5 slope (m/m), E 5 modulus of elasticity (Pa 5 N/m2), I 5 moment of inertia (m4), M(x) 5 moment (N m), V(x) 5 shear (N), and w(x) 5 distributed load (N/m). For the case of a linearly increas- ing load (recall Fig. P8.18), the slope can be computed analyti- cally as

u(x) 5 w0

120EIL (25x4 1 6L2x2 2 L4) (P24.19.1)

Employ (a) numerical integration to compute the defl ection (in m) and (b) numerical differentiation to compute the moment (in N m) and shear (in N). Base your numerical calculations on values of the slope computed with Eq. P24.19 at equally-spaced intervals of Dx 5 0.125 m along a 3-m beam. Use the following parameter values in your computation: E 5 200 GPa, I 5 0.0003 m4, and w0 5 2.5 kN/cm. In addition, the defl ections at the ends of the beam are set at y(0) 5 y(L) 5 0. Be careful of units. 24.20 You measure the following defl ections along the length of a simply-supported uniform beam (see Prob. 24.19)

Civil/Environmental Engineering 24.14 Perform the same computation as in Sec. 24.2, but use O(h8) Romberg integration to evaluate the integral. 24.15 Perform the same computation as in Sec. 24.2, but use Gauss quadrature to evaluate the integral. 24.16 As in Sec. 24.2, compute F using the trapezoidal rule and Simpson’s 1y3 and Simpson’s 3y8 rules but use the following force. Divide the mast into 5-ft intervals.

F 5 # 30

0

250z

6 1 z e2zy10 dz

24.17 Stream cross-sectional areas (A) are required for a number of tasks in water resources engineering, including fl ood forecasting and reservoir designing. Unless electronic sounding devices are available to obtain continuous profi les of the channel bottom, the engineer must rely on discrete depth measurements to compute A. An example of a typical stream cross section is shown in Fig. P24.17. The data points represent locations where a boat was anchored and depth read- ings taken. Use two trapezoidal rule applications (h 5 4 and 2 m) and Simpson’s 1y3 rule (h 5 2 m) to estimate the cross-sectional area from these data. 24.18 As described in Prob. 24.17, the cross-sectional area of a channel can be computed as

Ac 5 # B

0 H(y) dy

where B 5 the total channel width (m), H 5 the depth (m), and y 5 distance from the bank (m). In a similar fashion, the average fl ow Q (m3/s) can be computed as

Q 5 # B

0 U(y)H(y) dy

2010

Water surface

1.8 2 4 4 6 4 3.6 3.4 2.8

6

4

2

D ep

th , m

0

Distance from left bank, m 0

FIGURE P24.17 A stream cross section.

x, m 0 0.375 0.75 1.125 1.5 1.875 2.25 2.625 3

y, cm 0 20.2571 20.9484 21.9689 23.2262 24.6414 26.1503 27.7051 29.275

cha9792x_ch24_673-693.indd Page 687 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 687 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

688 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

p(z) 5 rg(D 2 z) (P24.23.1)

where p(z) 5 pressure in pascals (or N/m2) exerted at an elevation z meters above the reservoir bottom; r 5 density of water, which for this problem is assumed to be a constant 103 kg/m3; g 5 acceleration due to gravity (9.8 m/s2); and D 5 elevation (in m) of the water surface above the reservoir bottom. According to Eq. (P24.23.1), pressure increases linearly with depth, as depicted in Fig. P24.23a. Omitting atmospheric pressure (because it works against both sides of the dam face and essentially cancels out), the total force ft can be determined by multiplying pressure times the area of the dam face (as shown in Fig. P24.23b). Because both pressure and area vary with elevation, the total force is obtained by evaluating

Employ numerical differentiation to compute the slope, the moment (in N m), the shear (in N) and the distributed load (in N/m). Use the following parameter values in your computation: E 5 200 GPa, and I 5 0.0003 m4. 24.21 A transportation engineering study requires the calculation of the total number of cars that pass through an intersection over a 24-h period. An individual visits the intersection at various times during the course of a day and counts the number of cars that pass through the intersection in a minute. Utilize the data summarized in Table P24.21, to estimate the total number of cars that pass through the intersection per day. (Be careful of units.) 24.22 A wind force distributed against the side of a skyscraper is measured as

TABLE P24.21 Traffi c fl ow rate (cars/min) for an intersection measured at various times within a 24-h period.

Time Rate Time Rate Time Rate

12:00 midnight 2 9:00 A.M. 11 6:00 P.M. 20 2:00 A.M. 2 10:30 A.M. 4 7:00 P.M. 10 4:00 A.M. 0 11:30 A.M. 11 8:00 P.M. 8 5:00 A.M. 2 12:30 P.M. 12 9:00 P.M. 10 6:00 A.M. 6 2:00 P.M. 8 10:00 P.M. 8 7:00 A.M. 7 4:00 P.M. 7 11:00 P.M. 7 8:00 A.M. 23 5:00 P.M. 26 12:00 midnight 3

Height, l, m 0 30 60 90 120 150 180 210 240

Force, F(l ), N/m 0 340 1200 1600 2700 3100 3200 3500 3800

FIGURE P24.23 Water exerting pressure on the upstream face of a dam: (a) side view showing force increasing linearly with depth; (b) front view showing width of dam in meters.

0

(a) (b)

60

40

20

122 130 135 160 175 190 200

Compute the net force and the line of action due to this distributed wind. 24.23 Water exerts pressure on the upstream face of a dam as shown in Fig. P24.23. The pressure can be characterized by

ft 5 # D

0 rgw(z) (D 2 z) dz

where w(z) 5 width of the dam face (m) at elevation z (Fig. P24.23b). The line of action can also be obtained by evaluating

cha9792x_ch24_673-693.indd Page 688 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 688 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 689

As(z) 5 2 dV

dz (z)

where V 5 volume (m3) and z 5 depth (m) as measured from the surface down to the bottom. The average concentration of a sub- stance that varies with depth c (g/m3) can be computed by integration

c 5 #

Z

0 c(z)As(z) dz

# Z

0 As(z) dz

where Z 5 the total depth (m). Determine the average concentra- tion based on the following data:

z, m 0 4 8 12 16

V, 106 m3 9.8175 5.1051 1.9635 0.3927 0.0000

c, g/m3 10.2 8.5 7.4 5.2 4.1

Electrical Engineering 24.28 Perform the same computation as in Sec. 24.3, but for the current as specifi ed by

i(t) 5 5e21.25t sin 2pt  for 0 # t # Ty2 i(t) 5 0  for Ty2 , t # T

where T 5 1 s. Use fi ve-point Gauss quadrature to estimate the integral. 24.29 Repeat Prob. 24.28, but use fi ve applications of Simpson’s 1y3 rule. 24.30 Repeat Prob. 24.28, but use Romberg integration to es 5 1%. 24.31 Faraday’s law characterizes the voltage drop across an in- ductor as

VL 5 L di

dt

where VL 5 voltage drop (V), L 5 inductance (in henrys; 1 H 5 1 V ? s/A), i 5 current (A), and t 5 time (s). Determine the voltage drop as a function of time from the following data for an inductance of 4 H.

t 0 0.1 0.2 0.3 0.5 0.7

i 0 0.16 0.32 0.56 0.84 2.0

24.32 Based on Faraday’s law (Prob. 24.31), use the following voltage data to estimate the inductance in henrys if a current of 2 A is passed through the inductor over 400 milliseconds.

t, ms 0 10 20 40 60 80 120 180 280 400

V, volts 0 18 29 44 49 46 35 26 15 7

24.33 Suppose that the current through a resistor is described by the function

i(t) 5 (60 2 t)2 1 (60 2 t) sin(1t)

d 5 #

D

0 rgzw(z) (D 2 z) dz

# D

0 rgw(z) (D 2 z) dz

Use Simpson’s rule to compute ft and d. Check the results with your computer program for the trapezoidal rule. 24.24 To estimate the size of a new dam, you have to determine the total volume of water (m3) that fl ows down a river in a year’s time. You have available the following long-term average data for the river:

Date Mid- Jan.

Mid- Feb.

Mid- Mar.

Mid- Apr.

Mid- June

Mid- Sept.

Mid- Oct.

Mid- Nov.

Mid- Dec.

Flow, m3/s 30 38 82 125 95 20 22 24 35

Determine the volume. Be careful of units, and take care to make a proper estimate of fl ow at the end points. 24.25 The data listed in the following table gives hourly measure- ments of heat fl ux q (cal/cm2/h) at the surface of a solar collector. As an architectural engineer, you must estimate the total heat ab- sorbed by a 150,000-cm2 collector panel during a 14-h period. The panel has an absorption effi ciency eab of 45%. The total heat ab- sorbed is given by

h 5 eab # t

0 q A dt

where A 5 area and q 5 heat fl ux.

t 0 2 4 6 8 10 12 14

q 0.10 5.32 7.80 8.00 8.03 6.27 3.54 0.20

24.26 The heat fl ux q is the quantity of heat fl owing through a unit area of a material per unit time. It can be computed with Fourier’s law,

J 5 2k dT

dx

where J has units of J/m2/s or W/m2 and k is a coeffi cient of ther- mal conductivity that parameterizes the heat-conducting proper- ties of the material and has units of Wy(8C ? m). T 5 temperature (8C); and x 5 distance (m) along the path of heat fl ow. Fourier’s law is used routinely by architectural engineers to determine heat fl ow through walls. The following temperatures are measured from the surface (x 5 0) into a stone wall:

x, cm 0 0.08 0.16

T, °C 20 17 15

If the fl ux at x 5 0 is 60 W/m2, compute k. 24.27 The horizontal surface area As (m

2) of a lake at a particular depth can be computed from volume by differentiation,

cha9792x_ch24_673-693.indd Page 689 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 689 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

690 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

24.40 The rate of cooling of a body (Fig. P24.40) can be expressed as

dT

dt 5 2k(T 2 Ta)

where T 5 temperature of the body (8C), Ta 5 temperature of the surrounding medium (8C), and k 5 a proportionality constant (per minute). Thus, this equation (called Newton’s law of cool- ing) specifi es that the rate of cooling is proportional to the differ- ence in the temperatures of the body and of the surrounding medium. If a metal ball heated to 808C is dropped into water that is held constant at Ta 5 208C, the temperature of the ball changes, as in

Time, min 0 5 10 15 20 25

T, 8C 80 44.5 30.0 24.1 21.7 20.7

Utilize numerical differentiation to determine dTydt at each value of time. Plot dTydt versus T 2 Ta and employ linear regression to evaluate k. 24.41 A rod subject to an axial load (Fig. P24.41a) will be de- formed, as shown in the stress-strain curve in Fig. P24.41b. The area under the curve from zero stress out to the point of rupture is called the modulus of toughness of the material. It provides a measure of the energy per unit volume required to cause the mate- rial to rupture. As such, it is representative of the material’s ability to withstand an impact load. Use numerical integration to com- pute the modulus of toughness for the stress-strain curve seen in Fig. P24.41b. 24.42 If the velocity distribution of a fl uid fl owing through a pipe is known (Fig. P24.42), the fl ow rate Q (that is, the volume of water passing through the pipe per unit time) can be computed by Q 5 ey dA, where y is the velocity and A is the pipe’s cross- sectional area. (To grasp the meaning of this relationship physically, recall the close connection between summation and integration.) For a circular pipe, A 5 pr2 and dA 5 2pr dr. Therefore,

Q 5 # r

0 y(2pr) dr

and the resistance is a function of the current,

R 5 10i 1 2i2y3

Compute the average voltage over t 5 0 to 60 using the multiple- segment Simpson’s 1y3 rule. 24.34 If a capacitor initially holds no charge, the voltage across it as a function of time can be computed as

V(t) 5 1

C # t

0 i(t) dt

If C 5 1025 farad, use the following current data to develop a plot of voltage versus time:

t, s 0 0.2 0.4 0.6 0.8 1 1.2

i, 1023 A 0.2 0.3683 0.3819 0.2282 0.0486 0.0082 0.1441

Mechanical/Aerospace Engineering 24.35 Perform the same computation as in Sec. 24.4, but use the following equations:

F(x) 5 1.6x 2 0.045x2

u(x) 5 0.8 1 0.125x 2 0.009x2 1 0.0002x3

Use 4-, 8-, and 16-segment trapezoidal rules to compute the integral. 24.36 Repeat Prob. 24.35, but use (a) Simpson’s 1y3 rule, (b) Rom- berg integration to es 5 0.5%, and (c) Gauss quadrature. 24.37 Compute work as described in Sec. 24.4, but use the follow- ing equations for F(x) and u(x):

F(x) 5 1.6x 2 0.045x2

u(x) 5 20.00055x3 1 0.0123x2 1 0.13x

The force is in newtons and the angle is in radians. Perform the integration from x 5 0 to 30 m. 24.38 As was done in Sec. 24.4, determine the work performed if a constant force of 1 N applied at an angle u results in the following displacements. Use the MATLAB function cumtrapz to deter- mine the cumulative work and plot the result versus u.

x, m 0 1 2.7 3.8 3.7 3 1.4

u, deg 0 30 60 90 120 150 180

24.39 The work done on an object is equal to the force times the distance moved in the direction of the force. The velocity of an object in the direction of a force is given by

y 5 4t  0 # t # 4

y 5 16 1 (4 2 t)2  4 # t # 14

where y 5 mys. Employ the multiple-application Simpson’s rule to determine the work if a constant force of 200 N is applied for all t.

T

Ta

FIGURE P24.40

cha9792x_ch24_673-693.indd Page 690 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 690 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 691

t, s 0 0.52 1.04 1.75 2.37 3.25 3.83

x, m 153 185 208 249 261 271 273

where x is the distance from the end of the carrier. Estimate (a) velocity (dxydt) and (b) acceleration (dyydt) using numerical differentiation. 24.45 Employ the multiple-application Simpson’s rule to evaluate the vertical distance traveled by a rocket if the vertical velocity is given by

y 5 11t2 2 5t  0 # t # 10

y 5 1100 2 5t  10 # t # 20

y 5 50t 1 2(t 2 20)2  20 # t # 30

In addition, use numerical differentiation to develop graphs of the acceleration (dvydt) and the jerk (d2vydt2) versus time for t 5 0 to 30. Note that the jerk is very important because it is highly corre- lated with injuries such as whiplash. 24.46 The upward velocity of a rocket can be computed by the following formula:

y 5 u ln a m0 m0 2 qt

b 2 gt where y 5 upward velocity, u 5 velocity at which fuel is expelled relative to the rocket, m0 5 initial mass of the rocket at time t 5 0,

where r is the radial distance measured outward from the center of the pipe. If the velocity distribution is given by

y 5 2 a1 2 r r0 b1y6

where r0 is the total radius (in this case, 3 cm), compute Q using the multiple-application trapezoidal rule. Discuss the results. 24.43 Using the following data, calculate the work done by stretch- ing a spring that has a spring constant of k 5 300 N/m to x 5 0.35 m:

F, 103N 0 0.01 0.028 0.046 0.063 0.082 0.11 0.13

x, m 0 0.05 0.10 0.15 0.20 0.25 0.30 0.35

24.44 A jet fi ghter’s position on an aircraft carrier’s runway was timed during landing:

FIGURE P24.41 (a) A rod under axial loading and (b) the resulting stress-strain curve where stress is in kips per square inch (103 lb/in2) and strain is dimensionless.

0

20

40

60

0.1

s, k

si (b)(a)

e

0.02 0.05 0.10 0.15 0.20 0.25

s

40.0 37.5 43.0 52.0 60.0 55.0

Rupture

0.2 e

Modulus of toughness

FIGURE P24.42

r A

cha9792x_ch24_673-693.indd Page 691 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 691 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

692 CASE STUDIES: NUMERICAL INTEGRATION AND DIFFERENTIATION

H # P

0 aV 2 T a 0V

0T b

P b

d P

V, L

P, atm T 5 350 K T 5 400 K T 5 450 K

0.1 220 250 282.5 5 4.1 4.7 5.23 10 2.2 2.5 2.7 20 1.35 1.49 1.55 25 1.1 1.2 1.24 30 0.90 0.99 1.03 40 0.68 0.75 0.78 45 0.61 0.675 0.7 50 0.54 0.6 0.62

q 5 fuel consumption rate, and g 5 downward acceleration of gravity (assumed constant 5 9.8 m/s2). If u 5 1800 m/s, m0 5 160,000 kg, and q 5 2500 kg/s, use six-segment trapezoidal and Simpson’s 1y3 rule, six-point Gauss quadrature, and O(h8) Romberg methods to determine how high the rocket will fl y in 30 s. In addition, use numerical differentiation to generate a graph of acceleration as a function of time. 24.47 Referring to the data from Problem 20.61, fi nd the strain rate using fi nite difference methods. Use second-order accurate derivative approximations and plot your results. Looking at the graph, it is apparent that there is some experimental startup error. Find the mean and standard deviation of the strain rate after eliminating the data points representing the experimental startup error. 24.48 Fully developed fl ow moving through a 40-cm diameter pipe has the following velocity profi le:

Radius, r, cm 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0

Velocity, v, m/s 0.914 0.890 0.847 0.795 0.719 0.543 0.427 0.204 0

Find the volume fl ow rate Q using the relationship Q 5 eR0 2pry dr, where r is the radial axis of the pipe, R is the radius of the pipe, and y is the velocity. Solve the problem using two different approaches. (a) Fit a polynomial curve to the velocity data and integrate

analytically. (b) Use multiple-application Simpson’s 1y3 rule to integrate. (c) Find the percent error using the integral of the polynomial fi t as

the more correct value. 24.49 Fully developed fl ow of a Bingham plastic fl uid moving through a 12-in diameter pipe has the given velocity profi le. The fl ow of a Bingham fl uid does not shear the center core, producing plug fl ow in the region around the centerline.

Radius, r, in 0 1 2 3 4 5 6

Velocity, v, ft/s 5.00 5.00 4.62 4.01 3.42 1.69 0.00

Find the total volume fl ow rate Q using the relationship Q 5 e r2r1 2pr y dr 1 yc Ac, where r is the radial axis of the pipe, y is the velocity, yc is the velocity at the core, and Ac is the cross- sectional area of the plug. Solve the problem using two different approaches. (a) Fit a polynomial curve to the noncore data and integrate. (b) Use multiple-application Simpson’s rule to integrate. (c) Find the percent error using the integral of the polynomial fi t as

the more correct value. 24.50 The enthalpy of a real gas is a function of pressure as described below. These data were taken for a real fl uid. Estimate the enthalpy of the fl uid at 400 K and 50 atm (evaluate the integral from 0.1 atm to 50 atm).

24.51 Given the data below, fi nd the isothermal work done on the gas as it is compressed from 23 L to 3 L (remember that W 5 2eV2V1 P dV ).

V, L 3 8 13 18 23

P, atm 12.5 3.5 1.8 1.4 1.2

(a) Find the work performed on the gas numerically, using the 1-, 2-, and 4-segment trapezoidal rule.

(b) Compute the ratios of the errors in these estimates and relate them to the error analysis of the multiple-application trapezoidal rule discussed in Chap. 21.

24.52 The Rosin-Rammler-Bennet (RRB) equation is used to de- scribe size distribution in fi ne dust. F(x) represents the cumulative mass of dust particles of diameter x and smaller. x9 and n9 are con- stants equal to 30 mm and 1.44, respectively. The mass density distribution f(x) or the mass of dust particles of a diameter x is found by taking the derivative of the cumulative distribution

F(x) 5 1 2 e2(xyx¿) n¿  f (x) 5

dF(x)

dx

(a) Numerically calculate the mass density distribution f(x) and graph both f(x) and the cumulative distribution F(x).

(b) Using your results from part (a), calculate the mode size of the mass density distribution—that is, the size at which the deriva- tive of f(x) is equal to zero.

(c) Find the surface area per mass of the dust Sm (cm 2/g) using

Sm 5 6 r

# q

dmin

f (x)

x dx

cha9792x_ch24_673-693.indd Page 692 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 692 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 693

(a) Determine the pressure drop for a 10-cm length tube for a vis- cous liquid (m 5 0.005 N ? s/m2, density 5 r 5 1 3 103 kg/m3) with a fl ow of 10 3 1026 m3/s and the following varying radii along its length,

x, cm 0 2 4 5 6 7 10

r, mm 2 1.35 1.34 1.6 1.58 1.42 2

(b) Compare your result with the pressure drop that would have occurred if the tube had a constant radius equal to the average radius.

(c) Determine the average Reynolds number for the tube to verify that fl ow is truly laminar (Re 5 ryDym , 2100 where y 5 velocity).

24.55 Velocity data for air are collected at different radii from the centerline of a circular 16-cm-diameter pipe as tabulated below:

r, cm 0 1.60 3.20 4.80 6.40 7.47 7.87 7.95 8

v, m/s 10 9.69 9.30 8.77 7.95 6.79 5.57 4.89 0

Use numerical integration to determine the mass fl ow rate, which can be computed as

# R

0 ry2pr dr

where r 5 density (5 1.2 kg/m3). Express your results in kg/s.

The equation is valid only for spherical particles. Assume a density r 5 1 g cm23 and a minimum diameter of dust included in the dis- tribution dmin of 1 mm. 24.53 For fl uid fl ow over a surface, the heat fl ux to the surface can be computed as

J 5 2k dT

dy

where J 5 heat fl ux (W/m2), k 5 thermal conductivity (W/m ? K), T 5 temperature (K), and y 5 distance normal to the surface (m). The following measurements are made for air fl owing over a fl at plate that is 200 cm long and 50 cm wide:

y, cm 0 1 3 5

T, K 900 480 270 200

If k 5 0.028 J/s ? m ? K, (a) determine the fl ux at the surface and (b) the heat transfer in watts. Note that 1 J 5 1 W ? s. 24.54 The pressure gradient for laminar fl ow through a constant radius tube is given by

dp

dx 5 2

8m Q

pr4

where p 5 pressure (N/m2), x 5 distance along the tube’s centerline (m), m 5 dynamic viscosity (N ? s/m2), Q 5 fl ow (m3/s), and r 5 radius (m).

cha9792x_ch24_673-693.indd Page 693 29/10/13 9:48 PM F-468 cha9792x_ch24_673-693.indd Page 693 29/10/13 9:48 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

694

EPILOGUE: PART SIX

PT6.4 TRADE-OFFS

Table PT6.4 provides a summary of the trade-offs involved in numerical integration or quadrature. Most of these methods are based on the simple physical interpretation of an integral as the area under a curve. These techniques are designed to evaluate the integral of two different cases: (1) a mathematical function and (2) discrete data in tabular form. The Newton-Cotes formulas are the primary methods discussed in Chap. 21. They are applicable to both continuous and discrete functions. Both closed and open versions of these formulas are available. The open forms, which have integration limits that extend beyond the range of the data, are rarely used for the evaluation of defi nite integrals. However, they have utility for the solution of ordinary differential equations and for evaluating improper integrals. The closed Newton-Cotes formulas are based on replacing a mathematical function or tabulated data by an interpolating polynomial that is easy to integrate. The simplest version is the trapezoidal rule, which is based on taking the area below a straight line joining adjacent values of the function. One way to improve the accuracy of the trapezoi- dal rule is to divide the integration interval from a to b into a number of segments and apply the method to each segment. Aside from applying the trapezoidal rule with fi ner segmentation, another way to obtain a more accurate estimate of the integral is to use higher-order polynomials to

TABLE PT6.4 Comparison of the characteristics of alternative methods for numerical integration. The comparisons are based on general experience and do not account for the behavior of special functions.

Data Points Data Points Required for Required for Truncation Programming Method One Application n Applications Error Application Effort Comments

Trapezoidal rule 2 n 1 1 .h3f 0(j) Wide Easy Simpson’s 1/3 rule 3 2n 1 1 .h5f (4)(j) Wide Easy Simpson’s rule 3 or 4 $3 .h5f (4)(j) Wide Easy (1/3 and 3/8) Higher-order $5 N/A .h7f (6)(j) Rare Easy Newton-Cotes Romberg integration 3 Requires f(x) Moderate Inappropriate for be known tabular data Gauss quadrature $2 N/A Requires f(x) Easy Inappropriate for be known tabular data

cha9792x_ep06_694-697.indd Page 694 29/10/13 9:53 PM F-468 cha9792x_ep06_694-697.indd Page 694 29/10/13 9:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT6.6 ADVANCED METHODS AND ADDITIONAL REFERENCES 695

connect the points. If a quadratic equation is employed, the result is Simpson’s 1y3 rule. If a cubic equation is used, the result is Simpson’s 3y8 rule. Because they are much more accurate than the trapezoidal rule, these formulas are usually preferred and multiple- application versions are available. For situations with an even number of segments, the multiple application of the 1y3 rule is recommended. For an odd number of segments, the 3y8 rule can be applied to the last three segments and the 1y3 rule to the remaining segments. Higher-order Newton-Cotes formulas are also available. However, they are rarely used in practice. Where high accuracy is required, Romberg integration, adaptive quadra- ture, and Gauss quadrature methods are available. It should be noted that these ap- proaches are usually of practical value only in cases where the function is available. These techniques are ill-suited for tabulated data.

PT6.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT6.5 summarizes important formulas presented in Part Six. This table can be consulted to quickly access important relationships and formulas.

PT6.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

Although we have reviewed a number of numerical integration techniques, there are other methods that have utility in engineering practice. For example, adaptive schemes for solving ordinary differential equations can be used to evaluate complicated integrals, as will be discussed in Chap. 25. Another method for obtaining integrals is to fi t cubic splines to the data. The result- ing cubic equations can be integrated easily (Forsythe et al., 1977). A similar approach is also sometimes used for differentiation. Finally, aside from the Gauss-Legendre for- mulas discussed in Sec. 22.3, there are a variety of other quadrature formulas. Carnahan, Luther, and Wilkes (1969) and Ralston and Rabinowitz (1978) summarize many of these approaches. In summary, the foregoing is intended to provide you with avenues for deeper ex- ploration of the subject. Additionally, all the above references describe basic techniques covered in Part Six. We urge you to consult these alternative sources to broaden your understanding of numerical methods for integration.

cha9792x_ep06_694-697.indd Page 695 29/10/13 9:53 PM F-468 cha9792x_ep06_694-697.indd Page 695 29/10/13 9:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

696 EPILOGUE: PART SIX

2 (b 2 a)3

12 f  –(j)

2 (b 2 a)3

12n2 f –

2 (b 2 a)5

2880 f (4)(j)

2 (b 2 a)5

180n4 f  (4)

2 (b 2 a)5

6480 f   (4)(j)

O(h2k)

.f (2n12)(j)

Trapezoidal rule

Multiple-application trapezoidal rule

Simpson’s 1/3 rule

Multiple-application Simpson’s 1/3 rule

Simpson’s 3/8 rule

Romberg integration

Gauss quadrature

l . (b 2 a) f (a) 1 f (b)

2

l . (b 2 a) f (x0) 1 2a

n21

i51 f (xi) 1 f (xn)

2n

l . (b 2 a) f (x0) 1 4f (x1) 1 f (x2)

6

l . (b 2 a)

f (x0) 1 4 a n21

i51, 3 f (xi) 1 2 a

n22

j52, 4 f (xj) 1 f (xn)

3n

l . (b 2 a) f (x0) 1 3f (x1) 1 3f (x2) 1 f (x3)

8

lj, k 5 4k21 lj11, k21 2 lj, k21

4k21 2 1

l . c0f (x0) 1 c1f (x1) 1 p 1 cn21f  (xn21)

TABLE PT6.5 Summary of important formulas presented in Part Six.

Method Formulation Graphic Interpretations Error

f (x)

xa b

f (x)

xa = x0 b = xn

f (x)

xa = x0 b = x2

f (x)

xa = x0 b = xn

f (x)

xa = x0 b = x3

f (x)

xx0 x1

lj, k21 lj, k

lj11, k21

cha9792x_ep06_694-697.indd Page 696 29/10/13 9:53 PM F-468 cha9792x_ep06_694-697.indd Page 696 29/10/13 9:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

This page intentionally left blank

PART SEVEN

cha9792x_p07_698-708.indd Page 698 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 698 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

699

PT7.1 MOTIVATION

In the fi rst chapter of this book, we derived the following equation based on Newton’s second law to compute the velocity y of a falling parachutist as a function of time t [recall Eq. (1.9)]:

dy

dt 5 g 2

c m

y (PT7.1)

where g is the gravitational constant, m is the mass, and c is a drag coeffi cient. Such equations, which are composed of an unknown function and its derivatives, are called differential equations. Equation (PT7.1) is sometimes referred to as a rate equation because it expresses the rate of change of a variable as a function of variables and pa- rameters. Such equations play a fundamental role in engineering because many physical phenomena are best formulated mathematically in terms of their rate of change. In Eq. (PT7.1), the quantity being differentiated, y, is called the dependent variable. The quantity with respect to which y is differentiated, t, is called the independent vari- able. When the function involves one independent variable, the equation is called an ordinary differential equation (or ODE). This is in contrast to a partial differential equa- tion (or PDE) that involves two or more independent variables. Differential equations are also classifi ed as to their order. For example, Eq. (PT7.1) is called a fi rst-order equation because the highest derivative is a fi rst derivative. A second-order equation would include a second derivative. For example, the equation describing the position x of a mass-spring system with damping is the second-order equation,

m d 2x

dt 2 1 c

dx

dt 1 kx 5 0 (PT7.2)

where c is a damping coeffi cient and k is a spring constant. Similarly, an nth-order equa- tion would include an nth derivative. Higher-order equations can be reduced to a system of fi rst-order equations. For Eq. (PT7.2), this is done by defi ning a new variable y, where

y 5 dx

dt (PT7.3)

which itself can be differentiated to yield

dy

dt 5

d 2x

dt2 (PT7.4)

ORDINARY DIFFERENTIAL EQUATIONS

cha9792x_p07_698-708.indd Page 699 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 699 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

700 ORDINARY DIFFERENTIAL EQUATIONS

Equations (PT7.3) and (PT7.4) can then be substituted into Eq. (PT7.2) to give

m dy

dt 1 cy 1 kx 5 0 (PT7.5)

or

dy

dt 5 2

cy 1 kx

m (PT7.6)

Thus, Eqs. (PT7.3) and (PT7.6) are a pair of fi rst-order equations that are equivalent to the original second-order equation. Because other nth-order differential equations can be similarly reduced, this part of our book focuses on the solution of fi rst-order equations. Some of the engineering applications in Chap. 28 deal with the solution of second-order ODEs by reduction to a pair of fi rst-order equations.

PT7.1.1 Noncomputer Methods for Solving ODEs

Without computers, ODEs are usually solved with analytical integration techniques. For example, Eq. (PT7.1) could be multiplied by dt and integrated to yield

y 5 # ag 2 cm yb dt (PT7.7) The right-hand side of this equation is called an indefi nite integral because the limits of integration are unspecifi ed. This is in contrast to the defi nite integrals discussed previously in Part Six [compare Eq. (PT7.7) with Eq. (PT6.6)]. An analytical solution for Eq. (PT7.7) is obtained if the indefi nite integral can be evaluated exactly in equation form. For example, recall that for the falling parachutist problem, Eq. (PT7.7) was solved analytically by Eq. (1.10) (assuming y 5 0 at t 5 0):

y(t) 5 gm

c (1 2 e2(cym)t) (1.10)

The mechanics of deriving such analytical solutions will be discussed in Sec. PT7.2. For the time being, the important fact is that exact solutions for many ODEs of practical importance are not available. As is true for most situations discussed in other parts of this book, numerical methods offer the only viable alternative for these cases. Because these numerical methods usually require computers, engineers in the precomputer era were somewhat limited in the scope of their investigations. One very important method that engineers and applied mathematicians developed to overcome this dilemma was linearization. A linear ordinary differential equation is one that fi ts the general form

an(x)y (n) 1 p 1 a1(x)y¿ 1 a0(x)y 5 f(x) (PT7.8)

where y(n) is the nth derivative of y with respect to x and the a’s and f ’s are specifi ed functions of x. This equation is called linear because there are no products or nonlinear functions of the dependent variable y and its derivatives. The practical importance of linear ODEs is that they can be solved analytically. In contrast, most nonlinear equations

cha9792x_p07_698-708.indd Page 700 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 700 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT7.1 MOTIVATION 701

cannot be solved exactly. Thus, in the precomputer era, one tactic for solving nonlinear equations was to linearize them. A simple example is the application of ODEs to predict the motion of a swinging pendulum (Fig. PT7.1). In a manner similar to the derivation of the falling parachutist problem, Newton’s second law can be used to develop the following differential equation (see Sec. 28.4 for the complete derivation):

d 2u

dt 2 1

g

l sin u 5 0 (PT7.9)

where u is the angle of displacement of the pendulum, g is the gravitational constant, and l is the pendulum length. This equation is nonlinear because of the term sin u. One way to obtain an analytical solution is to realize that for small displacements of the pendulum from equilibrium (that is, for small values of u),

sin u > u (PT7.10)

Thus, if it is assumed that we are interested only in cases where u is small, Eq. (PT7.10) can be substituted into Eq. (PT7.9) to give

d 2u

dt 2 1

g

l u 5 0 (PT7.11)

We have, therefore, transformed Eq. (PT7.9) into a linear form that is easy to solve analytically. Although linearization remains a very valuable tool for engineering problem solving, there are cases where it cannot be invoked. For example, suppose that we were interested in studying the behavior of the pendulum for large displacements from equilibrium. In such instances, numerical methods offer a viable option for obtaining solutions. Today, the widespread availability of computers places this option within reach of all practicing engineers.

PT7.1.2 ODEs and Engineering Practice

The fundamental laws of physics, mechanics, electricity, and thermodynamics are usually based on empirical observations that explain variations in physical properties and states of systems. Rather than describing the state of physical systems directly, the laws are usually couched in terms of spatial and temporal changes. Several examples are listed in Table PT7.1. These laws defi ne mechanisms of change. When combined with continuity laws for energy, mass, or momentum, differential equa- tions result. Subsequent integration of these differential equations results in mathematical functions that describe the spatial and temporal state of a system in terms of energy, mass, or velocity variations. The falling parachutist problem introduced in Chap. 1 is an example of the derivation of an ordinary differential equation from a fundamental law. Recall that Newton’s second law was used to develop an ODE describing the rate of change of velocity of a falling parachutist. By integrating this relationship, we obtained an equation to predict fall veloc- ity as a function of time (Fig. PT7.2). This equation could be utilized in a number of different ways, including design purposes.

FIGURE PT7.1 The swinging pedulum.

l

cha9792x_p07_698-708.indd Page 701 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 701 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

702 ORDINARY DIFFERENTIAL EQUATIONS

In fact, such mathematical relationships are the basis of the solution for a great number of engineering problems. However, as described in the previous section, many of the differential equations of practical signifi cance cannot be solved using the analyti- cal methods of calculus. Thus, the methods discussed in the following chapters are extremely important in all fi elds of engineering.

TABLE PT7.1 Examples of fundamental laws that are written in terms of the rate of change of variables (t 5 time and x 5 position).

Law Mathematical Expression Variables and Parameters

Newton’s second law Velocity (v), force (F), and of motion mass (m)

Fourier’s heat law Heat fl ux (q), thermal conductivity (k9) and temperature (T)

Fick’s law of diffusion Mass fl ux ( J), diffusion coeffi cient (D), and concentration (c)

Faraday’s law Voltage drop (DVL), inductance (L), (voltage drop across and current (i) an inductor)

dv dt

5 F m

q 5 2k¿ dT dx

J 5 2D dc dx

¢VL 5 L di dt

F = ma

Analytical Numerical

v = (1 – e– (c/m)t)gmc vi + 1 = vi + (g – vi)�t c m

= g – vdv dt

c m

Physical law

Solution

ODE

FIGURE PT7.2 The sequence of events in the application of ODEs for engineering problem solving. The exam- ple shown is the velocity of a falling parachutist.

cha9792x_p07_698-708.indd Page 702 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 702 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT7.2 MATHEMATICAL BACKGROUND 703

PT7.2 MATHEMATICAL BACKGROUND

A solution of an ordinary differential equation is a specifi c function of the independent variable and parameters that satisfi es the original differential equation. To illustrate this concept, let us start with a given function

y 5 20.5x4 1 4x3 2 10x2 1 8.5x 1 1 (PT7.12)

which is a fourth-order polynomial (Fig. PT7.3a). Now, if we differentiate Eq. (PT7.12), we obtain an ODE:

dy

dx 5 22x3 1 12x2 2 20x 1 8.5 (PT7.13)

This equation also describes the behavior of the polynomial, but in a manner different from Eq. (PT7.12). Rather than explicitly representing the values of y for each value of x, Eq. (PT7.13) gives the rate of change of y with respect to x (that is, the slope) at every value of x. Figure PT7.3 shows both the function and the derivative plotted versus x. Notice

FIGURE PT7.3 Plots of (a) y versus x and (b) dy/dx versus x for the function y 5 20.5x4 1 4x3 2 10x2 1 8.5x 1 1.

y

4

(a) x3

dy/dx 8

(b)

x

– 8

3

cha9792x_p07_698-708.indd Page 703 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 703 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

704 ORDINARY DIFFERENTIAL EQUATIONS

how the zero values of the derivatives correspond to the point at which the original func- tion is fl at—that is, has a zero slope. Also, the maximum absolute values of the derivatives are at the ends of the interval where the slopes of the function are greatest. Although, as just demonstrated, we can determine a differential equation given the original function, the object here is to determine the original function given the differ- ential equation. The original function then represents the solution. For the present case, we can determine this solution analytically by integrating Eq. (PT7.13):

y 5 # (22x3 1 12x2 2 20x 1 8.5) dx Applying the integration rule (recall Table PT6.2)

#un du 5 u n11

n 1 1 1 C   n ? 21

to each term of the equation gives the solution

y 5 20.5x4 1 4x3 2 10x2 1 8.5x 1 C (PT7.14)

which is identical to the original function with one notable exception. In the course of differentiating and then integrating, we lost the constant value of 1 in the original equa- tion and gained the value C. This C is called a constant of integration. The fact that such an arbitrary constant appears indicates that the solution is not unique. In fact, it is but one of an infi nite number of possible functions (corresponding to an infi nite number of possible values of C) that satisfy the differential equation. For example, Fig. PT7.4 shows six possible functions that satisfy Eq. (PT7.14).

FIGURE PT7.4 Six possible solutions for the integral of 22x3 1 12x2 2 20x 1 8.5. Each conforms to a different value of the constant of integration C.

y

x C = 0

C = – 1

C = – 2

C = 3

C = 2

C = 1

cha9792x_p07_698-708.indd Page 704 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 704 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT7.3 ORIENTATION 705

Therefore, to specify the solution completely, a differential equation is usually ac- companied by auxiliary conditions. For fi rst-order ODEs, a type of auxiliary condition called an initial value is required to determine the constant and obtain a unique solution. For example, Eq. (PT7.13) could be accompanied by the initial condition that at x 5 0, y 5 1. These values could be substituted into Eq. (PT7.14):

1 5 20.5(0)4 1 4(0)3 2 10(0)2 1 8.5(0) 1 C (PT7.15)

to determine C 5 1. Therefore, the unique solution that satisfi es both the differential equation and the specifi ed initial condition is obtained by substituting C 5 1 into Eq. (PT7.14) to yield

y 5 20.5x4 1 4x3 2 10x2 1 8.5x 1 1 (PT7.16)

Thus, we have “pinned down’’ Eq. (PT7.14) by forcing it to pass through the initial condition, and in so doing, we have developed a unique solution to the ODE and have come full circle to the original function [Eq. (PT7.12)]. Initial conditions usually have very tangible interpretations for differential equations derived from physical problem settings. For example, in the falling parachutist problem, the initial condition was refl ective of the physical fact that at time zero the vertical veloc- ity was zero. If the parachutist had already been in vertical motion at time zero, the solution would have been modifi ed to account for this initial velocity. When dealing with an nth-order differential equation, n conditions are required to obtain a unique solution. If all conditions are specifi ed at the same value of the indepen- dent variable (for example, at x or t 5 0), then the problem is called an initial-value problem. This is in contrast to boundary-value problems where specifi cation of conditions occurs at different values of the independent variable. Chapters 25 and 26 will focus on initial-value problems. Boundary-value problems are covered in Chap. 27 along with eigenvalues.

PT7.3 ORIENTATION

Before proceeding to numerical methods for solving ordinary differential equations, some orientation might be helpful. The following material is intended to provide you with an overview of the material discussed in Part Seven. In addition, we have formulated objec- tives to focus your studies of the subject area.

PT7.3.1 Scope and Preview

Figure PT7.5 provides an overview of Part Seven. Two broad categories of numerical methods for initial-value problems will be discussed in this part of this book. One-step methods, which are covered in Chap. 25, permit the calculation of yi11, given the dif- ferential equation and yi. Multistep methods, which are covered in Chap. 26, require additional values of y other than at i. With all but a minor exception, the one-step methods in Chap. 25 belong to what are called Runge-Kutta techniques. Although the chapter might have been organized around this theoretical notion, we have opted for a more graphical, intuitive approach to introduce the methods. Thus, we begin the chapter with Euler’s method, which has a very straightforward graphical interpretation. Then, we use visually oriented arguments

cha9792x_p07_698-708.indd Page 705 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 705 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

706 ORDINARY DIFFERENTIAL EQUATIONS

FIGURE PT7.5 Schematic representation of the organization of Part Seven: Ordinary Differential Equations.

CHAPTER 25 Runge-Kutta

Methods

PART 7 Ordinary

Differential Equations

CHAPTER 26 Stiffness/ Multistep Methods

CHAPTER 27 Boundary Value and Eigenvalue

Problems

CHAPTER 28 Case Studies

EPILOGUE

26.2 Multistep methods

26.1 Stiffness

PT 7.2 Mathematical background

PT 7.6 Advanced methods

PT 7.5 Important formulas

28.4 Mechanical engineering

28.3 Electrical

engineering

28.2 Civil

engineering 28.1

Chemical engineering

27.1 Boundary-

value problems

27.3 Software packages

27.2 Eigenvalues

PT 7.4 Trade-offs

PT 7.3 Orientation

PT 7.1 Motivation

25.2 Heun and midpoint methods

25.1 Euler's method

25.3 Runge-Kutta

25.4 Systems of

ODEs

25.5 Adaptive RK

methods

cha9792x_p07_698-708.indd Page 706 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 706 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT7.3 ORIENTATION 707

to develop two improved versions of Euler’s method—the Heun and the midpoint tech- niques. After this introduction, we formally develop the concept of Runge-Kutta (or RK) approaches and demonstrate how the foregoing techniques are actually fi rst- and second- order RK methods. This is followed by a discussion of the higher-order RK formulations that are frequently used for engineering problem solving. In addition, we cover the ap- plication of one-step methods to systems of ODEs. Finally, the chapter ends with a discussion of adaptive RK methods that automatically adjust the step size in response to the truncation error of the computation. Chapter 26 starts with a description of stiff ODEs. These are both individual and systems of ODEs that have both fast and slow components to their solution. We intro- duce the idea of an implicit solution technique as one commonly used remedy for this problem. Next, we discuss multistep methods. These algorithms retain information of previous steps to more effectively capture the trajectory of the solution. They also yield the trunca- tion error estimates that can be used to implement step-size control. In this section, we initially take a visual, intuitive approach by using a simple method—the non-self-starting Heun—to introduce all the essential features of the multistep approaches. In Chap. 27 we turn to boundary-value and eigenvalue problems. For the former, we introduce both shooting and fi nite-difference methods. For the latter, we discuss sev- eral approaches, including the polynomial and the power methods. Finally, the chapter concludes with a description of the application of several software packages and librar- ies for solution of ODEs and eigenvalues. Chapter 28 is devoted to applications from all the fi elds of engineering. Finally, a short review section is included at the end of Part Seven. This epilogue summarizes and compares the important formulas and concepts related to ODEs. The comparison includes a discussion of trade-offs that are relevant to their implementation in engineering prac- tice. The epilogue also summarizes important formulas and includes references for advanced topics.

PT7.3.2 Goals and Objectives

Study Objectives. After completing Part Seven, you should have greatly enhanced your capability to confront and solve ordinary differential equations and eigenvalue prob- lems. General study goals should include mastering the techniques, having the capability to assess the reliability of the answers, and being able to choose the “best’’ method (or methods) for any particular problem. In addition to these general objectives, the specifi c study objectives in Table PT7.2 should be mastered.

Computer Objectives. Algorithms are provided for many of the methods in Part Seven. This information will allow you to expand your software library. For example, you may fi nd it useful from a professional viewpoint to have software that employs the fourth-order Runge-Kutta method for more than fi ve equations and to solve ODEs with an adaptive step-size approach. In addition, one of your most important goals should be to master several of the general-purpose software packages that are widely available. In particular, you should become adept at using these tools to implement numerical methods for engineering problem solving.

cha9792x_p07_698-708.indd Page 707 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 707 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

708 ORDINARY DIFFERENTIAL EQUATIONS

TABLE PT7.2 Specifi c study objectives for Part Seven.

1. Understand the visual representations of Euler’s, Heun’s, and the midpoint methods 2. Know the relationship of Euler’s method to the Taylor series expansion and the insight it provides

regarding the error of the method 3. Understand the difference between local and global truncation errors and how they relate to the

choice of a numerical method for a particular problem 4. Know the order and the step-size dependency of the global truncation errors for all the methods

described in Part Seven; understand how these errors bear on the accuracy of the techniques 5. Understand the basis of predictor-corrector methods; in particular, realize that the effi ciency of the

corrector is highly dependent on the accuracy of the predictor 6. Know the general form of the Runge-Kutta methods; understand the derivation of the second-order

RK method and how it relates to the Taylor series expansion; realize that there are an infi nite number of possible versions for second- and higher-order RK methods

7. Know how to apply any of the RK methods to systems of equations; be able to reduce an nth-order ODE to a system of n fi rst-order ODEs

8. Recognize the type of problem context where step size adjustment is important 9. Understand how adaptive step size control is integrated into a fourth-order RK method 10. Recognize how the combination of slow and fast components makes an equation or a system of

equations stiff 11. Understand the distinction between explicit and implicit solution schemes for ODEs; in particular,

recognize how the latter (1) ameliorates the stiffness problem and (2) complicates the solution mechanics

12. Understand the difference between initial-value and boundary-value problems 13. Know the difference between multistep and one-step methods; realize that all multistep methods are

predictor-correctors but that not all predictor-correctors are multistep methods 14. Understand the connection between integration formulas and predictor-corrector methods 15. Recognize the fundamental difference between Newton-Cotes and Adams integration formulas 16. Know the rationale behind the polynomial and the power methods for determining eigenvalues; in

particular, recognize their strengths and limitations 17. Understand how Hoteller’s defl ation allows the power method to be used to compute intermediate

eigenvalues 18. Know how to use software packages and/or libraries to integrate ODEs and evaluate eigenvalues

cha9792x_p07_698-708.indd Page 708 29/10/13 9:56 PM F-468 cha9792x_p07_698-708.indd Page 708 29/10/13 9:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25

709

C H A P T E R 25

Runge-Kutta Methods

This chapter is devoted to solving ordinary differential equations of the form

dy

dx 5 f(x, y)

In Chap. 1, we used a numerical method to solve such an equation for the velocity of the falling parachutist. Recall that the method was of the general form

New value 5 old value 1 slope 3 step size

or, in mathematical terms,

yi11 5 yi 1 fh (25.1)

According to this equation, the slope estimate of f is used to extrapolate from an old value yi to a new value yi 1 1 over a distance h (Fig. 25.1). This formula can be applied step by step to compute out into the future and, hence, trace out the trajectory of the solution.

FIGURE 25.1 Graphical depiction of a one- step method.

y

x

Step size = h

Slope = �

xi xi + 1

yi + 1 = yi + �h

cha9792x_ch25_709-754.indd Page 709 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 709 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

710 RUNGE-KUTTA METHODS

All one-step methods can be expressed in this general form, with the only difference being the manner in which the slope is estimated. As in the falling parachutist problem, the simplest approach is to use the differential equation to estimate the slope in the form of the fi rst derivative at xi. In other words, the slope at the beginning of the interval is taken as an approximation of the average slope over the whole interval. This approach, called Euler’s method, is discussed in the fi rst part of this chapter. This is followed by other one-step methods that employ alternative slope estimates that result in more ac- curate predictions. All these techniques are generally called Runge-Kutta methods.

25.1 EULER’S METHOD

The fi rst derivative provides a direct estimate of the slope at xi (Fig. 25.2):

f 5 f(xi, yi)

where f(xi, yi) is the differential equation evaluated at xi and yi. This estimate can be substituted into Eq. (25.1):

yi11 5 yi 1 f(xi, yi)h (25.2)

This formula is referred to as Euler’s (or the Euler-Cauchy or the point-slope) method. A new value of y is predicted using the slope (equal to the fi rst derivative at the original value of x) to extrapolate linearly over the step size h (Fig. 25.2).

EXAMPLE 25.1 Euler’s Method

Problem Statement. Use Euler’s method to numerically integrate Eq. (PT7.13):

dy

dx 5 22x3 1 12x2 2 20x 1 8.5

y

xxi + 1

error Predicted

True

xi

h

FIGURE 25.2 Euler’s method.

cha9792x_ch25_709-754.indd Page 710 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 710 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.1 EULER’S METHOD 711

from x 5 0 to x 5 4 with a step size of 0.5. The initial condition at x 5 0 is y 5 1. Recall that the exact solution is given by Eq. (PT7.16):

y 5 20.5x4 1 4x3 2 10x2 1 8.5x 1 1

Solution. Equation (25.2) can be used to implement Euler’s method:

y(0.5) 5 y(0) 1 f(0, 1)0.5

where y(0) 5 1 and the slope estimate at x 5 0 is

f(0, 1) 5 22(0)3 1 12(0)2 2 20(0) 1 8.5 5 8.5

Therefore,

y(0.5) 5 1.0 1 8.5(0.5) 5 5.25

The true solution at x 5 0.5 is

y 5 20.5(0.5)4 1 4(0.5)3 2 10(0.5)2 1 8.5(0.5) 1 1 5 3.21875

Thus, the error is

Et 5 true 2 approximate 5 3.21875 2 5.25 5 22.03125

or, expressed as percent relative error, et 5 263.1%. For the second step,

y(1) 5 y(0.5) 1 f(0.5, 5.25)0.5

5 5.25 1 [22(0.5)3 1 12(0.5)2 2 20(0.5) 1 8.5]0.5

5 5.875

The true solution at x 5 1.0 is 3.0, and therefore, the percent relative error is 295.8%. The computation is repeated, and the results are compiled in Table 25.1 and Fig. 25.3.

TABLE 25.1 Comparison of true and approximate values of the integral of y9 5 22x3 1 12x2 2 20x 1 8.5, with the initial condition that y 5 1 at x 5 0. The approximate values were computed using Euler’s method with a step size of 0.5. The local error refers to the error incurred over a single step. It is calculated with a Taylor series expansion as in Example 25.2. The global error is the total discrepancy due to past as well as present steps.

Percent Relative Error

x ytrue yEuler Global Local

0.0 1.00000 1.00000 0.5 3.21875 5.25000 263.1 263.1 1.0 3.00000 5.87500 295.8 228.1 1.5 2.21875 5.12500 2131.0 21.4 2.0 2.00000 4.50000 2125.0 20.3 2.5 2.71875 4.75000 274.7 17.2 3.0 4.00000 5.87500 246.9 3.9 3.5 4.71875 7.12500 251.0 211.3 4.0 3.00000 7.00000 2133.3 253.1

cha9792x_ch25_709-754.indd Page 711 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 711 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

712 RUNGE-KUTTA METHODS

Note that, although the computation captures the general trend of the true solution, the error is considerable. As discussed in the next section, this error can be reduced by using a smaller step size.

The preceding example uses a simple polynomial for the differential equation to facilitate the error analyses that follow. Thus,

dy

dx 5 f(x)

Obviously, a more general (and more common) case involves ODEs that depend on both x and y,

dy

dx 5 f(x, y)

As we progress through this part of the text, our examples will increasingly involve ODEs that depend on both the independent and the dependent variables.

25.1.1 Error Analysis for Euler’s Method

The numerical solution of ODEs involves two types of error (recall Chaps. 3 and 4):

1. Truncation, or discretization, errors caused by the nature of the techniques employed to approximate values of y.

FIGURE 25.3 Comparison of the true solution with a numerical solution using Euler’s method for the integral of y9 5 22x3 1 12x2 2 20x 1 8.5 from x 5 0 to x 5 4 with a step size of 0.5. The initial condition at x 5 0 is y 5 1.

y

4

0 x4

True solution

h = 0.5

20

cha9792x_ch25_709-754.indd Page 712 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 712 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.1 EULER’S METHOD 713

2. Round-off errors caused by the limited numbers of signifi cant digits that can be retained by a computer.

The truncation errors are composed of two parts. The fi rst is a local truncation error that results from an application of the method in question over a single step. The second is a propagated truncation error that results from the approximations produced during the previous steps. The sum of the two is the total, or global truncation, error. Insight into the magnitude and properties of the truncation error can be gained by deriving Euler’s method directly from the Taylor series expansion. To do this, realize that the differential equation being integrated will be of the general form

y¿ 5 f(x, y) (25.3)

where y9 5 dyydx and x and y are the independent and the dependent variables, respectively. If the solution—that is, the function describing the behavior of y—has continuous deriva- tives, it can be represented by a Taylor series expansion about a starting value (xi, yi), as in [recall Eq. (4.7)]

yi11 5 yi 1 y¿i h 1 y–i 2!

h2 1 p 1 y(n)i n!

hn 1 Rn (25.4)

where h 5 xi11 2 xi and Rn 5 the remainder term, defi ned as

Rn 5 y(n11)(j)

(n 1 1)! hn11 (25.5)

where j lies somewhere in the interval from xi to xi11. An alternative form can be de- veloped by substituting Eq. (25.3) into Eqs. (25.4) and (25.5) to yield

yi11 5 yi 1 f(xi, yi)h 1 f ¿(xi, yi)

2! h2 1 p 1

f (n21)(xi, yi)

n! hn 1 O(hn11) (25.6)

where O(hn11) specifi es that the local truncation error is proportional to the step size raised to the (n 1 1)th power. By comparing Eqs. (25.2) and (25.6), it can be seen that Euler’s method corresponds to the Taylor series up to and including the term f(xi, yi)h. Additionally, the comparison indicates that a truncation error occurs because we approximate the true solution using a fi nite number of terms from the Taylor series. We thus truncate, or leave out, a part of the true solution. For example, the truncation error in Euler’s method is attributable to the remaining terms in the Taylor series expansion that were not included in Eq. (25.2). Subtracting Eq. (25.2) from Eq. (25.6) yields

Et 5 f ¿(xi, yi)

2! h2 1 p 1 O(hn11) (25.7)

where Et 5 the true local truncation error. For suffi ciently small h, the errors in the terms in Eq. (25.7) usually decrease as the order increases (recall Example 4.2 and the ac- companying discussion), and the result is often represented as

Ea 5 f ¿(xi, yi)

2! h2 (25.8)

cha9792x_ch25_709-754.indd Page 713 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 713 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

714 RUNGE-KUTTA METHODS

or

Ea 5 O(h2) (25.9)

where Ea 5 the approximate local truncation error.

EXAMPLE 25.2 Taylor Series Estimate for the Error of Euler’s Method

Problem Statement. Use Eq. (25.7) to estimate the error of the fi rst step of Example 25.1. Also use it to determine the error due to each higher-order term of the Taylor series expansion.

Solution. Because we are dealing with a polynomial, we can use the Taylor series to obtain exact estimates of the errors in Euler’s method. Equation (25.7) can be written as

Et 5 f ¿(xi, yi)

2! h2 1

f –(xi, yi) 3!

h3 1 f (3)(xi, yi)

4! h4 (E25.2.1)

where f 9(xi, yi) 5 the fi rst derivative of the differential equation (that is, the second de- rivative of the solution). For the present case, this is

f ¿(xi, yi) 5 26x2 1 24x 2 20 (E25.2.2)

and f 0(xi, yi) is the second derivative of the ODE

f –(xi, yi) 5 212x 1 24 (E25.2.3)

and f (3)(xi, yi) is the third derivative of the ODE

f (3)(xi, yi) 5 212 (E25.2.4)

We can omit additional terms (that is, fourth derivatives and higher) from Eq. (E25.2.1) because for this particular case they equal zero. It should be noted that for other func- tions (for example, transcendental functions such as sinusoids or exponentials) this would not necessarily be true, and higher-order terms would have nonzero values. However, for the present case, Eqs. (E25.2.1) through (E25.2.4) completely defi ne the truncation error for a single application of Euler’s method. For example, the error due to truncation of the second-order term can be calculated as

Et, 2 5 26(0.0)2 1 24(0.0) 2 20

2 (0.5)2 5 22.5 (E25.2.5)

For the third-order term:

Et, 3 5 212(0.0) 1 24

6 (0.5)3 5 0.5

and the fourth-order term:

Et, 4 5 212

24 (0.5)4 5 20.03125

These three results can be added to yield the total truncation error:

Et 5 Et, 2 1 Et, 3 1 Et, 4 5 22.5 1 0.5 2 0.03125 5 22.03125

cha9792x_ch25_709-754.indd Page 714 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 714 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.1 EULER’S METHOD 715

which is exactly the error that was incurred in the initial step of Example 25.1. Note how Et, 2 . Et, 3 . Et, 4, which supports the approximation represented by Eq. (25.8).

As illustrated in Example 25.2, the Taylor series provides a means of quantifying the error in Euler’s method. However, there are limitations associated with its use for this purpose:

1. The Taylor series provides only an estimate of the local truncation error—that is, the error created during a single step of the method. It does not provide a measure of the propagated and, hence, the global truncation error. In Table 25.1, we have included the local and global truncation errors for Example 25.1. The local error was computed for each time step with Eq. (25.2) but using the true value of yi (the second column of the table) to compute each yi1l rather than the approximate value (the third column), as is done in the Euler method. As expected, the average absolute local truncation error (25 percent) is less than the average global error (90 percent). The only reason that we can make these exact error calculations is that we know the true value a priori. Such would not be the case in an actual problem. Consequently, as discussed below, you must usually apply techniques such as Euler’s method using a number of different step sizes to obtain an indirect estimate of the errors involved.

2. As mentioned above, in actual problems we usually deal with functions that are more complicated than simple polynomials. Consequently, the derivatives that are needed to evaluate the Taylor series expansion would not always be easy to obtain.

Although these limitations preclude exact error analysis for most practical problems, the Taylor series still provides valuable insight into the behavior of Euler’s method. Ac- cording to Eq. (25.9), we see that the local error is proportional to the square of the step size and the fi rst derivative of the differential equation. It can also be demonstrated that the global truncation error is O(h), that is, it is proportional to the step size (Carnahan et al., 1969). These observations lead to some useful conclusions:

1. The error can be reduced by decreasing the step size. 2. The method will provide error-free predictions if the underlying function (that is, the

solution of the differential equation) is linear, because for a straight line the second derivative would be zero.

This latter conclusion makes intuitive sense because Euler’s method uses straight-line segments to approximate the solution. Hence, Euler’s method is referred to as a fi rst- order method. It should also be noted that this general pattern holds for the higher-order one-step methods described in the following pages. That is, an nth-order method will yield perfect results if the underlying solution is an nth-order polynomial. Further, the local truncation error will be O(hn11) and the global error O(hn).

EXAMPLE 25.3 Effect of Reduced Step Size on Euler’s Method

Problem Statement. Repeat the computation of Example 25.1 but use a step size of 0.25.

cha9792x_ch25_709-754.indd Page 715 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 715 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

716 RUNGE-KUTTA METHODS

Solution. The computation is repeated, and the results are compiled in Fig. 25.4a. Halving the step size reduces the absolute value of the average global error to 40 percent and the absolute value of the local error to 6.4 percent. This is compared to global and local errors for Example 25.1 of 90 percent and 24.8 percent, respectively. Thus, as expected, the local error is quartered and the global error is halved. Also, notice how the local error changes sign for intermediate values along the range. This is due primarily to the fact that the fi rst derivative of the differential equation is a parabola that changes sign [recall Eq. (E25.2.2) and see Fig. 25.4b]. Because the local error is proportional to this function, the net effect of the oscillation in sign is to keep the global error from continuously growing as the calculation proceeds. Thus, from x 5 0 to x 5 1.25, the local errors are all negative, and consequently, the global error increases

FIGURE 25.4 (a) Comparison of two numerical solutions with Euler’s method using step sizes of 0.5 and 0.25. (b) Comparison of true and estimated local truncation error for the case where the step size is 0.5. Note that the “estimated” error is based on Eq. (E25.2.5).

y

4

0 x4

True solution

h = 0.5

h = 0.25

2 (a)

0

y

– 0.5

0 x42

True

Estimated

(b)

cha9792x_ch25_709-754.indd Page 716 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 716 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.1 EULER’S METHOD 717

over this interval. In the intermediate section of the range, positive local errors begin to reduce the global error. Near the end, the process is reversed and the global error again infl ates. If the local error continuously changes sign over the computation interval, the net effect is usually to reduce the global error. However, where the local errors are of the same sign, the numerical solution may diverge farther and farther from the true solution as the computation proceeds. Such results are said to be unstable.

The effect of further step-size reductions on the global truncation error of Euler’s method is illustrated in Fig. 25.5. This plot shows the absolute percent relative error at x 5 5 as a function of step size for the problem we have been examining in Examples 25.1 through 25.3. Notice that even when h is reduced to 0.001, the error still exceeds 0.1 percent. Because this step size corresponds to 5000 steps to proceed from x 5 0 to x 5 5, the plot suggests that a fi rst-order technique such as Euler’s method demands great computational effort to obtain acceptable error levels. Later in this chapter, we present higher-order techniques that attain much better accuracy for the same computa- tional effort. However, it should be noted that, despite its ineffi ciency, the simplicity of Euler’s method makes it an extremely attractive option for many engineering problems. Because it is very easy to program, the technique is particularly useful for quick analy- ses. In the next section, a computer algorithm for Euler’s method is developed.

FIGURE 25.5 Effect of step size on the global truncation error of Euler’s method for the integral of y9 5 22x3 1 12x2 2 20x 1 8.5. The plot shows the absolute percent relative global error at x 5 5 as a function of step size.

1

10

100

0.1 0.1

Step size

A b

so lu

te p

er ce

n t

re la

ti ve

e rr

o r

0.01 0.0011

50 Steps

500 50005

cha9792x_ch25_709-754.indd Page 717 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 717 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

718 RUNGE-KUTTA METHODS

25.1.2 Algorithm for Euler’s Method

Algorithms for one-step techniques such as Euler’s method are extremely simple to program. As specifi ed previously at the beginning of this chapter, all one-step methods have the general form

New value 5 old value 1 slope 3 step size (25.10)

The only way in which the methods differ is in the calculation of the slope. Suppose that you want to perform the simple calculation outlined in Table 25.1. That is, you would like to use Euler’s method to integrate y9 5 22x3 1 12x2 2 20x 1 8.5, with the initial condition that y 5 1 at x 5 0. You would like to integrate out to x 5 4 using a step size of 0.5, and display all the results. A simple pseudocode to accomplish this task could be written as in Fig. 25.6. Although this program will “do the job” of duplicating the results of Table 25.1, it is not very well designed. First, and foremost, it is not very modular. Although this is not very important for such a small program, it would be critical if we desired to mod- ify and improve the algorithm. Further, there are a number of issues related to the way we have set up the iterations. For example, suppose that the step size were to be made very small to obtain better ac- curacy. In such cases, because every computed value is displayed, the number of output values might be very large. Further, the algorithm is predicated on the assumption that the calculation interval is evenly divisible by the step size. Finally, the accumulation of x in the line x 5 x 1 dx can be subject to quantizing errors of the sort previously dis-

FIGURE 25.6 Pseudocode for a “dumb” version of Euler’s method.

‘set integration range xi 5 0 xf 5 4 ‘initialize variables x 5 xi y 5 1 ‘set step size and determine ‘number of calculation steps dx 5 0.5 nc 5 (xf 2 xi)/dx ‘output initial condition PRINT x, y ‘loop to implement Euler’s method ‘and display results DOFOR i 5 1, nc dydx 5 22x3 1 12x2 2 20x 1 8.5 y 5 y 1 dydx ? dx x 5 x 1 dx PRINT x, y END DO

cha9792x_ch25_709-754.indd Page 718 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 718 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.1 EULER’S METHOD 719

cussed in Sec. 3.4.1. For example, if dx were changed to 0.01 and standard IEEE fl oat- ing point representation were used (about seven signifi cant digits), the result at the end of the calculation would be 3.999997 rather than 4. For dx 5 0.001, it would be 3.999892! A much more modular algorithm that avoids these diffi culties is displayed in Fig. 25.7. The algorithm does not output all calculated values. Rather, the user specifi es an output interval, xout, that dictates the interval at which calculated results are stored in arrays, xpm and ypm. These values are stored in arrays so that they can be output in a variety of ways after the computation is completed (for example, printed, graphed, or written to a fi le). The Driver Program takes big output steps and calls an Integrator routine that takes fi ner calculation steps. Note that the loops controlling both large and small steps exit on logical conditions. Thus, the intervals do not have to be evenly divisible by the step sizes. The Integrator routine then calls an Euler routine that takes a single step with Euler’s method. The Euler routine calls a Derivative routine that calculates the derivative value. Whereas such modularization might seem like overkill for the present case, it will greatly facilitate modifying the program in later sections. For example, although the program in Fig. 25.7 is specifi cally designed to implement Euler’s method, the Euler module is the only part that is method-specifi c. Thus, all that is required to apply this algorithm to the other one-step methods is to modify this routine.

FIGURE 25.7 Pseudocode for an “improved” modular version of Euler’s method.

(a) Main or “Driver” Program

Assign values for y 5 initial value dependent variable xi 5 initial value independent variable xf 5 final value independent variable dx 5 calculation step size xout 5 output interval

x 5 xi m 5 0 xpm 5 x ypm 5 y DO xend 5 x 1 xout IF (xend . xf) THEN xend 5 xf h 5 dx CALL Integrator (x, y, h, xend) m 5 m 1 1 xpm 5 x ypm 5 y IF (x $ xf) EXIT END DO DISPLAY RESULTS END

(b) Routine to Take One Output Step

SUB Integrator (x, y, h, xend) DO IF (xend 2 x , h) THEN h 5 xend 2 x CALL Euler (x, y, h, ynew) y 5 ynew IF (x $ xend) EXIT END DO END SUB

(c) Euler’s Method for a Single ODE

SUB Euler (x, y, h, ynew) CALL Derivs (x, y, dydx) ynew 5 y 1 dydx * h x 5 x 1 h END SUB

(d) Routine to Determine Derivative

SUB Derivs (x, y, dydx) dydx 5 . . . END SUB

cha9792x_ch25_709-754.indd Page 719 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 719 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

720 RUNGE-KUTTA METHODS

EXAMPLE 25.4 Solving ODEs with the Computer

Problem Statement. A computer program can be developed from the pseudocode in Fig. 25.7. We can use this software to solve another problem associated with the falling parachutist. You recall from Part One that our mathematical model for the velocity was based on Newton’s second law in the form

dy

dt 5 g 2

c m

y (E25.4.1)

This differential equation was solved both analytically (Example 1.1) and numerically using Euler’s method (Example 1.2). These solutions were for the case where g 5 9.81, c 5 12.5, m 5 68.1, and y 5 0 at t 5 0. The objective of the present example is to repeat these numerical computations employing a more complicated model for the velocity based on a more complete math- ematical description of the drag force caused by wind resistance. This model is given by

dy

dt 5 g 2

c m

cy 1 a a y ymax bb d (E25.4.2)

where g, m, and c are the same as for Eq. (E25.4.1), and a, b, and ymax are empirical constants, which for this case are equal to 8.3, 2.2, and 46, respectively. Note that this model is more capable of accurately fi tting empirical measurements of drag forces versus velocity than is the simple linear model of Example 1.1. However, this increased fl exibil- ity is gained at the expense of evaluating three coeffi cients rather than one. Furthermore, the resulting mathematical model is more diffi cult to solve analytically. In this case, Euler’s method provides a convenient alternative to obtain an approximate numerical solution.

FIGURE 25.8 Graphical results for the solution of the nonlinear ODE [Eq. (E25.4.2)]. Notice that the plot also shows the solution for the linear model [Eq. (E25.4.1)] for comparative purposes.

60

y

40

20

0 t15

Nonlinear

Linear

5 100

cha9792x_ch25_709-754.indd Page 720 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 720 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.2 IMPROVEMENTS OF EULER’S METHOD 721

Solution. The results for both the linear and nonlinear model are displayed in Fig. 25.8 with an integration step size of 0.1 s. The plot in Fig. 25.8 also shows an overlay of the solution of the linear model for comparison purposes. The results of the two simulations indicate how increasing the complexity of the for- mulation of the drag force affects the velocity of the parachutist. In this case, the terminal velocity is lowered because of resistance caused by the higher-order terms in Eq. (E25.4.2). Alternative models could be tested in a similar fashion. The combination of a com- puter-generated solution makes this an easy and effi cient task. This convenience should allow you to devote more of your time to considering creative alternatives and holistic aspects of the problem rather than to tedious manual computations.

25.1.3 Higher-Order Taylor Series Methods

One way to reduce the error of Euler’s method would be to include higher-order terms of the Taylor series expansion in the solution. For example, including the second-order term from Eq. (25.6) yields

yi11 5 yi 1 f(xi, yi)h 1 f ¿(xi, yi)

2! h2 (25.11)

with a local truncation error of

Ea 5 f –(xi, yi)

6 h3

Although the incorporation of higher-order terms is simple enough to implement for polynomials, their inclusion is not so trivial when the ODE is more complicated. In particular, ODEs that are a function of both the dependent and independent variable require chain-rule differentiation. For example, the fi rst derivative of f(x, y) is

f ¿(xi, yi) 5 0f(x, y) 0x

1 0f(x, y) 0y

dy

dx

The second derivative is

f –(xi, yi) 5 0[0fy0x 1 (0fy0y)(dyydx)]

0x 1 0[0fy0x 1 (0fy0y)(dyydx)]

0y dy

dx

Higher-order derivatives become increasingly more complicated. Consequently, as described in the following sections, alternative one-step methods have been developed. These schemes are comparable in performance to the higher-order Taylor-series approaches but require only the calculation of fi rst derivatives.

25.2 IMPROVEMENTS OF EULER’S METHOD

A fundamental source of error in Euler’s method is that the derivative at the beginning of the interval is assumed to apply across the entire interval. Two simple modifi cations are available to help circumvent this shortcoming. As will be demonstrated in Sec. 25.3, both modifi cations actually belong to a larger class of solution techniques called Runge-Kutta

cha9792x_ch25_709-754.indd Page 721 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 721 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

722 RUNGE-KUTTA METHODS

methods. However, because they have a very straightforward graphical interpretation, we will present them prior to their formal derivation as Runge-Kutta methods.

25.2.1 Heun’s Method

One method to improve the estimate of the slope involves the determination of two derivatives for the interval—one at the initial point and another at the end point. The two derivatives are then averaged to obtain an improved estimate of the slope for the entire interval. This approach, called Heun’s method, is depicted graphically in Fig. 25.9. Recall that in Euler’s method, the slope at the beginning of an interval

y¿i 5 f(xi, yi) (25.12)

is used to extrapolate linearly to yi11:

y0i11 5 yi 1 f(xi, yi)h (25.13)

For the standard Euler method we would stop at this point. However, in Heun’s method the y0i11 calculated in Eq. (25.13) is not the fi nal answer, but an intermediate prediction. This is why we have distinguished it with a superscript 0. Equation (25.13) is called a

FIGURE 25.9 Graphical depiction of Heun’s method. (a) Predictor and (b) corrector.

y

xxi + 1xi

(a)

Slope = f (xi, yi)

Slope = f (xi + 1, y 0 i + 1)

y

xxi + 1xi

(b)

Slope = f (xi, yi) + f (xi + 1, yi + 1) 2

0

cha9792x_ch25_709-754.indd Page 722 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 722 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.2 IMPROVEMENTS OF EULER’S METHOD 723

predictor equation. It provides an estimate of yi11 that allows the calculation of an esti- mated slope at the end of the interval:

y¿i11 5 f(xi11, y0i11) (25.14)

Thus, the two slopes [Eqs. (25.12) and (25.14)] can be combined to obtain an average slope for the interval:

y¿ 5 y¿i 1 y¿i11

2 5

f(xi, yi) 1 f(xi11, y0i11) 2

This average slope is then used to extrapolate linearly from yi to yi 1 l using Euler’s method:

yi11 5 yi 1 f(xi, yi) 1 f(xi11, y0i11)

2 h

which is called a corrector equation. The Heun method is a predictor-corrector approach. All the multistep methods to be discussed subsequently in Chap. 26 are of this type. The Heun method is the only one-step predictor-corrector method described in this book. As derived above, it can be expressed concisely as

Predictor (Fig. 25.9a): y0i11 5 yi 1 f(xi, yi)h (25.15)

Corrector (Fig. 25.9b): yi11 5 yi 1 f(xi, yi) 1 f(xi11, y0i11)

2 h (25.16)

Note that because Eq. (25.16) has yi1l on both sides of the equal sign, it can be applied in an iterative fashion. That is, an old estimate can be used repeatedly to provide an im- proved estimate of yi1l. The process is depicted in Fig. 25.10. It should be understood that

FIGURE 25.10 Graphical representation of iterating the corrector of Heun’s method to obtain an improved estimate.

f(xi , y i ) + f (x

i+1 , y i+

1 )

yi + h

y i +

1

2

0

cha9792x_ch25_709-754.indd Page 723 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 723 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

724 RUNGE-KUTTA METHODS

this iterative process does not necessarily converge on the true answer but will converge on an estimate with a fi nite truncation error, as demonstrated in the following example. As with similar iterative methods discussed in previous sections of the book, a ter- mination criterion for convergence of the corrector is provided by [recall Eq. (3.5)]

Zea Z 5 ` y j i11 2 y

j21 i11

y ji11 ` 100% (25.17)

where y j21i11 and y j i11 are the result from the prior and the present iteration of the correc-

tor, respectively.

EXAMPLE 25.5 Heun’s Method

Problem Statement. Use Heun’s method to integrate y9 5 4e0.8x 2 0.5y from x 5 0 to x 5 4 with a step size of 1. The initial condition at x 5 0 is y 5 2.

Solution. Before solving the problem numerically, we can use calculus to determine the following analytical solution:

y 5 4

1.3 (e0.8x 2 e20.5x) 1 2e20.5x (E25.5.1)

This formula can be used to generate the true solution values in Table 25.2. First, the slope at (x0, y0) is calculated as

y¿0 5 4e0 2 0.5(2) 5 3

This result is quite different from the actual average slope for the interval from 0 to 1.0, which is equal to 4.1946, as calculated from the differential equation using Eq. (PT6.4). The numerical solution is obtained by using the predictor [Eq. (25.15)] to obtain an estimate of y at 1.0:

y01 5 2 1 3(1) 5 5

TABLE 25.2 Comparison of true and approximate values of the integral of y9 5 4e0.8x 2 0.5y, with the initial condition that y 5 2 at x 5 0. The approximate values were computed using the Heun method with a step size of 1. Two cases, corresponding to different numbers of corrector iterations, are shown, along with the absolute percent relative error.

Iterations of Heun’s Method

1 15

x ytrue y Heun |Et| (%) y Heun |Et| (%)

0 2.0000000 2.0000000 0.00 2.0000000 0.00 1 6.1946314 6.7010819 8.18 6.3608655 2.68 2 14.8439219 16.3197819 9.94 15.3022367 3.09 3 33.6771718 37.1992489 10.46 34.7432761 3.17 4 75.3389626 83.3377674 10.62 77.7350962 3.18

cha9792x_ch25_709-754.indd Page 724 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 724 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.2 IMPROVEMENTS OF EULER’S METHOD 725

Note that this is the result that would be obtained by the standard Euler method. The true value in Table 25.2 shows that it corresponds to a percent relative error of 19.3 percent. Now, to improve the estimate for yi11, we use the value y

0 1 to predict the slope at

the end of the interval

y¿1 5 f(x1, y01) 5 4e0.8(1) 2 0.5(5) 5 6.402164

which can be combined with the initial slope to yield an average slope over the interval from x 5 0 to 1

y¿ 5 3 1 6.402164

2 5 4.701082

which is closer to the true average slope of 4.1946. This result can then be substituted into the corrector [Eq. (25.16)] to give the prediction at x 5 1

y1 5 2 1 4.701082(1) 5 6.701082

which represents a percent relative error of 28.18 percent. Thus, the Heun method with- out iteration of the corrector reduces the absolute value of the error by a factor of 2.4 as compared with Euler’s method. Now this estimate can be used to refi ne or correct the prediction of y1 by substitut- ing the new result back into the right-hand side of Eq. (25.16):

y1 5 2 1 [3 1 4e0.8(1) 2 0.5(6.701082)]

2 1 5 6.275811

which represents an absolute percent relative error of 1.31 percent. This result, in turn, can be substituted back into Eq. (25.16) to further correct:

y1 5 2 1 [3 1 4e0.8(1) 2 0.5(6.275811)]

2 1 5 6.382129

which represents an Zet Z of 3.03%. Notice how the errors sometimes grow as the iterations proceed. Such increases can occur, especially for large step sizes, and they prevent us from drawing the general conclusion that an additional iteration will always improve the result. However, for a suffi ciently small step size, the iterations should eventually con- verge on a single value. For our case, 6.360865, which represents a relative error of 2.68 percent, is attained after 15 iterations. Table 25.2 shows results for the remainder of the computation using the method with 1 and 15 iterations per step.

In the previous example, the derivative is a function of both the dependent variable y and the independent variable x. For cases such as polynomials, where the ODE is solely a function of the independent variable, the predictor step [Eq. (25.16)] is not required and the corrector is applied only once for each iteration. For such cases, the technique is expressed concisely as

yi11 5 yi 1 f(xi) 1 f(xi11)

2 h (25.18)

cha9792x_ch25_709-754.indd Page 725 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 725 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

726 RUNGE-KUTTA METHODS

Notice the similarity between the right-hand side of Eq. (25.18) and the trapezoidal rule [Eq. (21.3)]. The connection between the two methods can be formally demonstrated by starting with the ordinary differential equation

dy

dx 5 f(x)

This equation can be solved for y by integration:

# yi11

yi dy 5 #

xi11

xi f(x) dx (25.19)

which yields

yi11 2 yi 5 # xi11

xi f(x) dx (25.20)

or

yi11 5 yi 1 # xi11

xi f(x) dx (25.21)

Now, recall from Chap. 21 that the trapezoidal rule [Eq. (21.3)] is defi ned as

# xi11

xi f(x) dx >

f(xi) 1 f(xi11)

2 h (25.22)

where h 5 xi11 2 xi. Substituting Eq. (25.22) into Eq. (25.21) yields

yi11 5 yi 1 f(xi) 1 f(xi11)

2 h (25.23)

which is equivalent to Eq. (25.18). Because Eq. (25.23) is a direct expression of the trapezoidal rule, the local truncation error is given by [recall Eq. (21.6)]

Et 5 2 f –(j)

12 h3 (25.24)

where j is between xi and xi1l. Thus, the method is second order because the second de- rivative of the ODE is zero when the true solution is a quadratic. In addition, the local and global errors are O(h3) and O(h2), respectively. Therefore, decreasing the step size decreases the error at a faster rate than for Euler’s method. Figure 25.11, which shows the result of using Heun’s method to solve the polynomial from Example 25.1 demonstrates this behavior.

25.2.2 The Midpoint (or Improved Polygon) Method

Figure 25.12 illustrates another simple modifi cation of Euler’s method. Called the mid- point method (or the improved polygon or the modifi ed Euler), this technique uses Euler’s method to predict a value of y at the midpoint of the interval (Fig. 25.12a):

yi11y2 5 yi 1 f(xi, yi) h

2 (25.25)

cha9792x_ch25_709-754.indd Page 726 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 726 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.2 IMPROVEMENTS OF EULER’S METHOD 727

FIGURE 25.11 Comparison of the true solution with a numerical solution using Euler’s and Heun’s methods for the integral of y9 5 22x3 1 12x2 2 20x 1 8.5.

y

5

x

True solution

Euler’s method

Heun’s method

3

y

xxi + 1xi

y Slope = f (xi + 1/2, yi + 1/2)

xxi + 1/2xi

(b)

(a)

Slope = f (xi + 1/2, yi + 1/2)

FIGURE 25.12 Graphical depiction of the midpoint method. (a) Eq. (25.25) and (b) Eq. (25.27).

cha9792x_ch25_709-754.indd Page 727 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 727 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

728 RUNGE-KUTTA METHODS

Then, this predicted value is used to calculate a slope at the midpoint:

y¿i11y2 5 f(xi11y2, yi11y2) (25.26)

which is assumed to represent a valid approximation of the average slope for the entire interval. This slope is then used to extrapolate linearly from xi to xi1l (Fig. 25.12b):

yi11 5 yi 1 f(xi11y2, yi11y2)h (25.27)

Observe that because yi1l is not on both sides, the corrector [Eq. (25.27)] cannot be ap- plied iteratively to improve the solution. As in the previous section, this approach can also be linked to Newton-Cotes inte- gration formulas. Recall from Table 21.4, that the simplest Newton-Cotes open integra- tion formula, which is called the midpoint method, can be represented as

# b

a f(x) dx > (b 2 a) f(x1)

where x1 is the midpoint of the interval (a, b). Using the nomenclature for the present case, it can be expressed as

# xi11

xi f(x) dx > h f(xi11y2)

Substitution of this formula into Eq. (25.21) yields Eq. (25.27). Thus, just as the Heun method can be called the trapezoidal rule, the midpoint method gets its name from the underlying integration formula upon which it is based. The midpoint method is superior to Euler’s method because it utilizes a slope estimate at the midpoint of the prediction interval. Recall from our discussion of numerical differ- entiation in Sec. 4.1.3 that centered fi nite divided differences are better approximations of derivatives than either forward or backward versions. In the same sense, a centered ap- proximation such as Eq. (25.26) has a local truncation error of O(h2) in comparison with the forward approximation of Euler’s method, which has an error of O(h). Consequently, the local and global errors of the midpoint method are O(h3) and O(h2), respectively.

25.2.3 Computer Algorithms for Heun and Midpoint Methods

Both the Heun method with a single corrector and the midpoint method can be easily programmed using the general structure depicted in Fig. 25.7. As in Fig. 25.13a and b, simple routines can be written to take the place of the Euler routine in Fig. 25.7. However, when the iterative version of the Heun method is to be implemented, the modifi cations are a bit more involved (although they are still localized within a single module). We have developed pseudocode for this purpose in Fig. 25.13c. This algorithm can be combined with Fig. 25.7 to develop software for the iterative Heun method.

25.2.4 Summary

By tinkering with Euler’s method, we have derived two new second-order techniques. Even though these versions require more computational effort to determine the slope, the accom- panying reduction in error will allow us to conclude in a subsequent section (Sec. 25.3.4)

cha9792x_ch25_709-754.indd Page 728 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 728 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.3 RUNGE-KUTTA METHODS 729

that the improved accuracy is usually worth the effort. Although there are certain cases where easily programmable techniques such as Euler’s method can be applied to advantage, the Heun and midpoint methods are generally superior and should be implemented if they are consistent with the problem objectives. As noted at the beginning of this section, the Heun (without iterations), the midpoint method, and in fact, the Euler technique itself are versions of a broader class of one-step approaches called Runge-Kutta methods. We now turn to a formal derivation of these techniques.

25.3 RUNGE-KUTTA METHODS

Runge-Kutta (RK) methods achieve the accuracy of a Taylor series approach without requiring the calculation of higher derivatives. Many variations exist but all can be cast in the generalized form of Eq. (25.1):

yi11 5 yi 1 f(xi, yi, h)h (25.28)

where f(xi, yi, h) is called an increment function, which can be interpreted as a represen- tative slope over the interval. The increment function can be written in general form as

f 5 a1k1 1 a2k2 1 p 1 ankn (25.29)

(a) Simple Heun without Iteration

SUB Heun (x, y, h, ynew) CALL Derivs (x, y, dy1dx) ye 5 y 1 dy1dx ? h CALL Derivs(x 1 h, ye, dy2dx) Slope 5 (dy1dx 1 dy2dx)y2 ynew 5 y 1 Slope ? h x 5 x 1 h END SUB

(b) Midpoint Method

SUB Midpoint (x, y, h, ynew) CALL Derivs(x, y, dydx) ym 5 y 1 dydx ? hy2 CALL Derivs (x 1 hy2, ym, dymdx) ynew 5 y 1 dymdx ? h x 5 x 1 h END SUB

(c) Heun with Iteration

SUB HeunIter (x, y, h, ynew) es 5 0.01 maxit 5 20 CALL Derivs(x, y, dy1dx) ye 5 y 1 dy1dx ? h iter 5 0 DO yeold 5 ye CALL Derivs(x 1 h, ye, dy2dx) slope 5 (dy1dx 1 dy2dx)y2 ye 5 y 1 slope ? h iter 5 iter 1 1

ea 5 ` ye 2 yeold ye

` 100% IF (ea # es OR iter . maxit) EXIT END DO ynew 5 ye x 5 x 1 h END SUB

FIGURE 25.13 Pseudocode to implement the (a) simple Heun, (b) midpoint, and (c) iterative Heun methods.

cha9792x_ch25_709-754.indd Page 729 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 729 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

730 RUNGE-KUTTA METHODS

where the a’s are constants and the k’s are

k1 5 f(xi, yi) (25.29a)

k2 5 f(xi 1 p1h, yi 1 q11k1h) (25.29b)

k3 5 f(xi 1 p2h, yi 1 q21k1h 1 q22k2h) (25.29c) # # #

kn 5 f(xi 1 pn21h, yi 1 qn21, 1k1h 1 qn21, 2k2h 1 p 1 qn21, n21kn21h) (25.29d)

where the p’s and q’s are constants. Notice that the k’s are recurrence relationships. That is, k1 appears in the equation for k2, which appears in the equation for k3, and so forth. Because each k is a functional evaluation, this recurrence makes RK methods effi cient for computer calculations. Various types of Runge-Kutta methods can be devised by employing different num- bers of terms in the increment function as specifi ed by n. Note that the fi rst-order RK method with n 5 1 is, in fact, Euler’s method. Once n is chosen, values for the a’s, p’s, and q’s are evaluated by setting Eq. (25.28) equal to terms in a Taylor series expansion (Box 25.1). Thus, at least for the lower-order versions, the number of terms, n, usually represents the order of the approach. For example, in the next section, second-order RK methods use an increment function with two terms (n 5 2). These second-order methods will be exact if the solution to the differential equation is quadratic. In addition, because terms with h3 and higher are dropped during the derivation, the local truncation error is O(h3) and the global error is O(h2). In subsequent sections, the third- and fourth-order RK methods (n 5 3 and 4, respectively) are developed. For these cases, the global trun- cation errors are O(h3) and O(h4), respectively.

25.3.1 Second-Order Runge-Kutta Methods

The second-order version of Eq. (25.28) is

yi11 5 yi 1 (a1k1 1 a2k2)h (25.30)

where

k1 5 f(xi, yi) (25.30a)

k2 5 f(xi 1 p1h, yi 1 q11k1h) (25.30b)

As described in Box 25.1, values for al, a2, p1, and q11 are evaluated by setting Eq. (25.30) equal to a Taylor series expansion to the second-order term. By doing this, we derive three equations to evaluate the four unknown constants. The three equations are

a1 1 a2 5 1 (25.31)

a2 p1 5 1

2 (25.32)

a2q11 5 1

2 (25.33)

cha9792x_ch25_709-754.indd Page 730 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 730 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.3 RUNGE-KUTTA METHODS 731

Because we have three equations with four unknowns, we must assume a value of one of the unknowns to determine the other three. Suppose that we specify a value for a2. Then Eqs. (25.31) through (25.33) can be solved simultaneously for

a1 5 1 2 a2 (25.34)

p1 5 q11 5 1

2a2 (25.35)

Box 25.1 Derivation of the Second-Order Runge-Kutta Methods

The second-order version of Eq. (25.28) is

yi11 5 yi 1 (a1k1 1 a2k2)h (B25.1.1)

where

k1 5 f (xi, yi) (B25.1.2)

and

k2 5 f (xi 1 p1h, yi 1 q11k1h) (B25.1.3)

To use Eq. (B25.1.1) we have to determine values for the constants a1, a2, p1, and q11. To do this, we recall that the second- order Taylor series for yi11 in terms of yi and f(xi, yi) is written as [Eq. (25.11)]

yi11 5 yi 1 f (xi, yi)h 1 f ¿(xi, yi)

2! h2 (B25.1.4)

where f 9(xi, yi) must be determined by chain-rule differentiation (Sec. 25.1.3):

f ¿(xi, yi) 5 0f (x, y) 0x

1 0f (x, y) 0y

dy

dx (B25.1.5)

Substituting Eq. (B25.1.5) into (B25.1.4) gives

yi11 5 yi 1 f (xi, yi)h 1 a 0f0x 1 0f 0y

dy

dx b h2

2! (B25.1.6)

The basic strategy underlying Runge-Kutta methods is to use alge- braic manipulations to solve for values of a1, a2, p1, and q11 that make Eqs. (B25.1.1) and (B25.1.6) equivalent. To do this, we fi rst use a Taylor series to expand Eq. (25.1.3). The Taylor series for a two-variable function is defi ned as [recall Eq. (4.26)]

g(x 1 r, y 1 s) 5 g(x, y) 1 r 0g 0x

1 s 0g 0y

1p

Applying this method to expand Eq. (B25.1.3) gives

f (xi 1 p1h, yi 1 q11k1h) 5 f (xi, yi) 1 p1h 0f 0x

1 q11k1h 0f 0y

1 O(h2)

This result can be substituted along with Eq. (B25.1.2) into Eq. (B25.1.1) to yield

yi11 5 yi 1 a1h f (xi, yi) 1 a2h f(xi, yi) 1 a2 p1h 2

0f 0x

1 a2q11h 2 f (xi, yi)

0f 0y

1 O(h3)

or, by collecting terms,

yi11 5 yi 1 [a1 f (xi, yi) 1 a2 f (xi, yi) ]h

1 c a2 p1 0f0x 1 a2q11 f (xi, yi) 0f 0y d h2 1 O(h3)

(B25.1.7)

Now, comparing like terms in Eqs. (B25.1.6) and (B25.1.7), we determine that for the two equations to be equivalent, the following must hold:

a1 1 a2 5 1

a2 p1 5 1

2

a2q11 5 1

2

These three simultaneous equations contain the four unknown con- stants. Because there is one more unknown than the number of equations, there is no unique set of constants that satisfy the equa- tions. However, by assuming a value for one of the constants, we can determine the other three. Consequently, there is a family of second-order methods rather than a single version.

cha9792x_ch25_709-754.indd Page 731 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 731 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

732 RUNGE-KUTTA METHODS

Because we can choose an infi nite number of values for a2, there are an infi nite number of second-order RK methods. Every version would yield exactly the same results if the solution to the ODE were quadratic, linear, or a constant. However, they yield different results when (as is typically the case) the solution is more complicated. We present three of the most commonly used and preferred versions:

Heun Method with a Single Corrector (a2 5 1y2). If a2 is assumed to be 1y2, Eqs. (25.34) and (25.35) can be solved for a1 5 1y2 and pl 5 q11 5 1. These parameters, when substituted into Eq. (25.30), yield

yi11 5 yi 1 a12 k1 1 1

2 k2b h (25.36)

where

k1 5 f(xi, yi) (25.36a)

k2 5 f(xi 1 h, yi 1 k1h) (25.36b)

Note that k1 is the slope at the beginning of the interval and k2 is the slope at the end of the interval. Consequently, this second-order Runge-Kutta method is actually Heun’s technique without iteration.

The Midpoint Method (a2 5 1). If a2 is assumed to be 1, then a1 5 0, p1 5 q11 5 1y2, and Eq. (25.30) becomes

yi11 5 yi 1 k 2h (25.37)

where

k1 5 f(xi, yi) (25.37a)

k2 5 f axi 1 12 h, yi 1 1

2 k1hb (25.37b)

This is the midpoint method.

Ralston’s Method (a2 5 2y3). Ralston (1962) and Ralston and Rabinowitz (1978) determined that choosing a2 5 2y3 provides a minimum bound on the truncation error for the second-order RK algorithms. For this version, a1 5 1y3 and p1 5 q11 5 3y4 and yields

yi11 5 yi 1 a13 k1 1 2

3 k2b h (25.38)

where

k1 5 f(xi, yi) (25.38a)

k2 5 f axi 1 34 h, yi 1 3

4 k1hb (25.38b)

cha9792x_ch25_709-754.indd Page 732 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 732 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.3 RUNGE-KUTTA METHODS 733

EXAMPLE 25.6 Comparison of Various Second-Order RK Schemes

Problem Statement. Use the midpoint method [Eq. (25.37)] and Ralston’s method [Eq. (25.38)] to numerically integrate Eq. (PT7.13)

f(x, y) 5 22x3 1 12x2 2 20x 1 8.5

from x 5 0 to x 5 4 using a step size of 0.5. The initial condition at x 5 0 is y 5 1. Compare the results with the values obtained using another second-order RK algorithm, that is, the Heun method without corrector iteration (Table 25.3).

Solution. The fi rst step in the midpoint method is to use Eq. (25.37a) to compute

k1 5 22(0) 3 1 12(0)2 2 20(0) 1 8.5 5 8.5

However, because the ODE is a function of x only, this result has no bearing on the second step—the use of Eq. (25.37b) to compute

k2 5 22(0.25) 3 1 12(0.25)2 2 20(0.25) 1 8.5 5 4.21875

Notice that this estimate of the slope is much closer to the average value for the interval (4.4375) than the slope at the beginning of the interval (8.5) that would have been used for Euler’s approach. The slope at the midpoint can then be substituted into Eq. (25.37) to predict

y(0.5) 5 1 1 4.21875(0.5) 5 3.109375  et 5 3.4%

The computation is repeated, and the results are summarized in Fig. 25.14 and Table 25.3.

FIGURE 25.14 Comparison of the true solution with numerical solutions using three second-order RK methods and Euler’s method.

y

4

0 x420

Analytical Euler Heun Midpoint Ralston

cha9792x_ch25_709-754.indd Page 733 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 733 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

734 RUNGE-KUTTA METHODS

For Ralston’s method, k1 for the fi rst interval also equals 8.5 and [Eq. (25.38b)]

k2 5 22(0.375) 3 1 12(0.375)2 2 20(0.375) 1 8.5 5 2.58203125

The average slope is computed by

f 5 1

3 (8.5) 1

2

3 (2.58203125) 5 4.5546875

which can be used to predict

y(0.5) 5 1 1 4.5546875(0.5) 5 3.27734375  et 5 21.82%

The computation is repeated, and the results are summarized in Fig. 25.14 and Table 25.3. Notice how all the second-order RK methods are superior to Euler’s method.

25.3.2 Third-Order Runge-Kutta Methods

For n 5 3, a derivation similar to the one for the second-order method can be performed. The result of this derivation is six equations with eight unknowns. Therefore, values for two of the unknowns must be specifi ed a priori in order to determine the remaining parameters. One common version that results is

yi11 5 yi 1 1

6 (k1 1 4k2 1 k3)h (25.39)

where

k1 5 f(xi, yi) (25.39a)

TABLE 25.3 Comparison of true and approximate values of the integral of y9 5 22x3 1 12x2 2 20x 1 8.5, with the initial condition that y 5 1 at x 5 0. The approximate values were computed using three versions of second-order RK methods with a step size of 0.5.

Second-Order Heun Midpoint Ralston RK

x ytrue y |Et| (%) y |Et| (%) y |Et| (%)

0.0 1.00000 1.00000 0 1.00000 0 1.00000 0 0.5 3.21875 3.43750 6.8 3.109375 3.4 3.277344 1.8 1.0 3.00000 3.37500 12.5 2.81250 6.3 3.101563 3.4 1.5 2.21875 2.68750 21.1 1.984375 10.6 2.347656 5.8 2.0 2.00000 2.50000 25.0 1.75 12.5 2.140625 7.0 2.5 2.71875 3.18750 17.2 2.484375 8.6 2.855469 5.0 3.0 4.00000 4.37500 9.4 3.81250 4.7 4.117188 2.9 3.5 4.71875 4.93750 4.6 4.609375 2.3 4.800781 1.7 4.0 3.00000 3.00000 0 3 0 3.031250 1.0

cha9792x_ch25_709-754.indd Page 734 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 734 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.3 RUNGE-KUTTA METHODS 735

k2 5 f axi 1 12 h, yi 1 1

2 k1hb (25.39b)

k3 5 f(xi 1 h, yi 2 k1h 1 2k2h) (25.39c)

Note that if the derivative is a function of x only, this third-order method reduces to Simpson’s 1y3 rule. Ralston (1962) and Ralston and Rabinowitz (1978) have developed an alternative version that provides a minimum bound on the truncation error. In any case, the third-order RK methods have local and global errors of O(h4) and O(h3), re- spectively, and yield exact results when the solution is a cubic. When dealing with polynomials, Eq. (25.39) will also be exact when the differential equation is cubic and the solution is quartic. This is because Simpson’s 1y3 rule provides exact integral esti- mates for cubics (recall Box 21.3).

25.3.3 Fourth-Order Runge-Kutta Methods

The most popular RK methods are fourth order. As with the second-order approaches, there are an infi nite number of versions. The following is the most commonly used form, and we therefore call it the classical fourth-order RK method:

yi11 5 yi 1 1

6 (k1 1 2k2 1 2k3 1 k4)h (25.40)

where

k1 5 f(xi, yi) (25.40a)

k2 5 f axi 1 12 h, yi 1 1

2 k1hb (25.40b)

FIGURE 25.15 Graphical depiction of the slope estimates comprising the fourth-order RK method.

y

xxi+1/2

h

xi

k2

k1

k3

k3

k2

k1

k4

xi+1

cha9792x_ch25_709-754.indd Page 735 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 735 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

736 RUNGE-KUTTA METHODS

k3 5 f axi 1 12 h, yi 1 1

2 k2hb (25.40c)

k4 5 f(xi 1 h, yi 1 k3h) (25.40d)

Notice that for ODEs that are a function of x alone, the classical fourth-order RK method is similar to Simpson’s 1y3 rule. In addition, the fourth-order RK method is similar to the Heun approach in that multiple estimates of the slope are developed in order to come up with an improved average slope for the interval. As depicted in Fig. 25.15, each of the k’s represents a slope. Equation (25.40) then represents a weighted average of these to arrive at the improved slope.

EXAMPLE 25.7 Classical Fourth-Order RK Method

Problem Statement.

(a) Use the classical fourth-order RK method [Eq. (25.40)] to integrate

f(x, y) 5 22x3 1 12x2 2 20x 1 8.5

using a step size of h 5 0.5 and an initial condition of y 5 1 at x 5 0. (b) Similarly, integrate

f(x, y) 5 4e0.8x 2 0.5y

using h 5 0.5 with y(0) 5 2 from x 5 0 to 0.5.

Solution.

(a) Equations (25.40a) through (25.40d) are used to compute k1 5 8.5, k2 5 4.21875, k3 5 4.21875 and k4 5 1.25, which are substituted into Eq. (25.40) to yield

y(0.5) 5 1 1 e 1 6

[8.5 1 2(4.21875) 1 2(4.21875) 1 1.25] f 0.5 5 3.21875

which is exact. Thus, because the true solution is a quartic [Eq. (PT7.16)], the fourth- order method gives an exact result.

(b) For this case, the slope at the beginning of the interval is computed as

k1 5 f(0, 2) 5 4e 0.8(0) 2 0.5(2) 5 3

This value is used to compute a value of y and a slope at the midpoint,

y(0.25) 5 2 1 3(0.25) 5 2.75

k2 5 f(0.25, 2.75) 5 4e 0.8(0.25) 2 0.5(2.75) 5 3.510611

This slope in turn is used to compute another value of y and another slope at the midpoint,

y(0.25) 5 2 1 3.510611(0.25) 5 2.877653

k3 5 f(0.25, 2.877653) 5 4e 0.8(0.25) 2 0.5(2.877653) 5 3.446785

Next, this slope is used to compute a value of y and a slope at the end of the interval,

y(0.5) 5 2 1 3.071785(0.5) 5 3.723392

k4 5 f(0.5, 3.723392) 5 4e 0.8(0.5) 2 0.5(3.723392) 5 4.105603

cha9792x_ch25_709-754.indd Page 736 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 736 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.3 RUNGE-KUTTA METHODS 737

Finally, the four slope estimates are combined to yield an average slope. This average slope is then used to make the fi nal prediction at the end of the interval.

f 5 1

6 [3 1 2(3.510611) 1 2(3.446785) 1 4.105603] 5 3.503399

y(0.5) 5 2 1 3.503399(0.5) 5 3.751699

which compares favorably with the true solution of 3.751521.

25.3.4 Higher-Order Runge-Kutta Methods

Where more accurate results are required, Butcher’s (1964) fi fth-order RK method is recommended:

yi11 5 yi 1 1

90 (7k1 1 32k3 1 12k4 1 32k5 1 7k6)h (25.41)

where

k1 5 f(xi, yi) (25.41a)

k2 5 f axi 1 14 h, yi 1 1

4 k1hb (25.41b)

k3 5 f axi 1 14 h, yi 1 1

8 k1h 1

1

8 k2hb (25.41c)

k4 5 f axi 1 12 h, yi 2 1

2 k2h 1 k3hb (25.41d)

k5 5 f axi 1 34 h, yi 1 3

16 k1h 1

9

16 k4hb (25.41e)

k6 5 f axi 1 h, yi 2 37 k1h 1 2

7 k2h 1

12

7 k3h 2

12

7 k4h 1

8

7 k5hb (25.41f)

Note the similarity between Butcher’s method and Boole’s rule in Table 21.2. Higher-order RK formulas such as Butcher’s method are available, but in general, beyond fourth-order methods the gain in accuracy is offset by the added computational effort and complexity.

EXAMPLE 25.8 Comparison of Runge-Kutta Methods

Problem Statement. Use fi rst- through fi fth-order RK methods to solve

f(x, y) 5 4e0.8x 2 0.5y

with y(0) 5 2 from x 5 0 to x 5 4 with various step sizes. Compare the accuracy of the various methods for the result at x 5 4 based on the exact answer of y(4) 5 75.33896.

Solution. The computation is performed using Euler’s, the noniterative Heun, the third- order RK [Eq. (25.39)], the classical fourth-order RK, and Butcher’s fi fth-order RK methods. The results are presented in Fig. 25.16, where we have plotted the absolute

cha9792x_ch25_709-754.indd Page 737 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 737 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

738 RUNGE-KUTTA METHODS

value of the percent relative error versus the computational effort. This latter quantity is equivalent to the number of function evaluations required to attain the result, as in

Effort 5 nf b 2 a

h (E25.8.1)

where nf 5 the number of function evaluations involved in the particular RK computa- tion. For orders # 4, nf is equal to the order of the method. However, note that Butcher’s fi fth-order technique requires six function evaluations [Eq. (25.41a) through (25.41f)]. The quantity (b 2 a)yh is the total integration interval divided by the step size—that is, it is the number of applications of the RK technique required to obtain the result. Thus, because the function evaluations are usually the primary time-consuming steps, Eq. (E25.8.1) provides a rough measure of the run time required to attain the answer. Inspection of Fig. 25.16 leads to a number of conclusions: fi rst, that the higher-order methods attain better accuracy for the same computational effort and, second, that the gain in accuracy for the additional effort tends to diminish after a point. (Notice that the curves drop rapidly at fi rst and then tend to level off.)

Example 25.8 and Fig. 25.16 might lead one to conclude that higher-order RK tech- niques are always the preferred methods. However, other factors such as programming

FIGURE 25.16 Comparison of percent relative error versus computational effort for fi rst- through fi fth-order RK methods.

100

1

10– 2

10– 4

10– 6

Euler

Heun

RK–3d

RK–4th

Butcher

Effort P

er ce

n t

re la

ti ve

e rr

o r

cha9792x_ch25_709-754.indd Page 738 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 738 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.4 SYSTEMS OF EQUATIONS 739

costs and the accuracy requirements of the problem also must be considered when choos- ing a solution technique. Such trade-offs will be explored in detail in the engineering applications in Chap. 28 and in the epilogue for Part Seven.

25.3.5 Computer Algorithms for Runge-Kutta Methods

As with all the methods covered in this chapter, the RK techniques fi t nicely into the general algorithm embodied in Fig. 25.7. Figure 25.17 presents pseudocode to determine the slope of the classic fourth-order RK method [Eq. (25.40)]. Subroutines to compute slopes for all the other versions can be easily programmed in a similar fashion.

25.4 SYSTEMS OF EQUATIONS

Many practical problems in engineering and science require the solution of a system of simultaneous ordinary differential equations rather than a single equation. Such systems may be represented generally as

dy1 dx

5 f1(x, y1, y2, p , yn)

dy2 dx

5 f2(x, y1, y2, p , yn)

# # #

dyn dx

5 fn(x, y1, y2, p , yn) (25.42)

The solution of such a system requires that n initial conditions be known at the starting value of x.

SUB RK4 (x, y, h, ynew) CALL Derivs(x, y, k1) ym 5 y 1 k1 ? hy2 CALL Derivs(x 1 hy2, ym, k2) ym 5 y 1 k2 ? hy2 CALL Derivs(x 1 hy2, ym, k3) ye 5 y 1 k3 ? h CALL Derivs(x 1 h, ye, k4) slope 5 (k1 1 2(k2 1 k3) 1 k4)y6 ynew 5 y 1 slope ? h x 5 x 1 h END SUB

FIGURE 25.17 Pseudocode to determine a single step of the fourth-order RK method.

cha9792x_ch25_709-754.indd Page 739 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 739 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

740 RUNGE-KUTTA METHODS

25.4.1 Euler’s Method

All the methods discussed in this chapter for single equations can be extended to the system shown above. Engineering applications can involve thousands of simultaneous equations. In each case, the procedure for solving a system of equations simply involves applying the one-step technique for every equation at each step before proceeding to the next step. This is best illustrated by the following example for the simple Euler’s method.

EXAMPLE 25.9 Solving Systems of ODEs Using Euler’s Method

Problem Statement. Solve the following set of differential equations using Euler’s method, assuming that at x 5 0, y1 5 4, and y2 5 6. Integrate to x 5 2 with a step size of 0.5.

dy1 dx

5 20.5y1   dy2 dx

5 4 2 0.3y2 2 0.1y1

Solution. Euler’s method is implemented for each variable as in Eq. (25.2):

y1(0.5) 5 4 1 [20.5(4)]0.5 5 3

y2(0.5) 5 6 1 [4 2 0.3(6) 2 0.1(4)]0.5 5 6.9

Note that y1(0) 5 4 is used in the second equation rather than the y1(0.5) 5 3 computed with the fi rst equation. Proceeding in a like manner gives

x y1 y2

0 4 6 0.5 3 6.9 1.0 2.25 7.715 1.5 1.6875 8.44525 2.0 1.265625 9.094087

25.4.2 Runge-Kutta Methods

Note that any of the higher-order RK methods in this chapter can be applied to systems of equations. However, care must be taken in determining the slopes. Figure 25.15 is help- ful in visualizing the proper way to do this for the fourth-order method. That is, we fi rst develop slopes for all variables at the initial value. These slopes (a set of k1’s) are then used to make predictions of the dependent variable at the midpoint of the interval. These midpoint values are in turn used to compute a set of slopes at the midpoint (the k2’s). These new slopes are then taken back to the starting point to make another set of midpoint pre- dictions that lead to new slope predictions at the midpoint (the k3’s). These are then em- ployed to make predictions at the end of the interval that are used to develop slopes at the end of the interval (the k4’s). Finally, the k’s are combined into a set of increment functions [as in Eq. (25.40)] and brought back to the beginning to make the fi nal prediction. The following example illustrates the approach.

cha9792x_ch25_709-754.indd Page 740 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 740 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.4 SYSTEMS OF EQUATIONS 741

EXAMPLE 25.10 Solving Systems of ODEs Using the Fourth-Order RK Method

Problem Statement. Use the fourth-order RK method to solve the ODEs from Ex- ample 25.9.

Solution. First, we must solve for all the slopes at the beginning of the interval:

k1,1 5 f1(0, 4, 6) 5 20.5(4) 5 22

k1, 2 5 f2(0, 4, 6) 5 4 2 0.3(6) 2 0.1(4) 5 1.8

where ki, j is the ith value of k for the jth dependent variable. Next, we must calculate the fi rst values of y1 and y2 at the midpoint:

y1 1 k1,1 h

2 5 4 1 (22)

0.5

2 5 3.5

y2 1 k1, 2 h

2 5 6 1 (1.8)

0.5

2 5 6.45

which can be used to compute the fi rst set of midpoint slopes,

k2, 1 5 f1(0.25, 3.5, 6.45) 5 21.75

k2, 2 5 f2(0.25, 3.5, 6.45) 5 1.715

These are used to determine the second set of midpoint predictions,

y1 1 k2,1 h

2 5 4 1 (21.75)

0.5

2 5 3.5625

y2 1 k2, 2 h

2 5 6 1 (1.715)

0.5

2 5 6.42875

which can be used to compute the second set of midpoint slopes,

k3, 1 5 f1(0.25, 3.5625, 6.42875) 5 21.78125

k3, 2 5 f2(0.25, 3.5625, 6.42875) 5 1.715125

These are used to determine the predictions at the end of the interval

y1 1 k3,1h 5 4 1 (21.78125)(0.5) 5 3.109375

y2 1 k3, 2h 5 6 1 (1.715125)(0.5) 5 6.857563

which can be used to compute the endpoint slopes,

k4,1 5 f1(0.5, 3.109375, 6.857563) 5 21.554688

k4, 2 5 f2(0.5, 3.109375, 6.857563) 5 1.631794

The values of k can then be used to compute [Eq. (25.40)]:

y1(0.5) 5 4 1 1

6 [22 1 2(21.75 2 1.78125) 2 1.554688]0.5 5 3.115234

y2(0.5) 5 6 1 1

6 [1.8 1 2(1.715 1 1.715125) 1 1.631794]0.5 5 6.857670

cha9792x_ch25_709-754.indd Page 741 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 741 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

742 RUNGE-KUTTA METHODS

Proceeding in a like manner for the remaining steps yields

x y1 y2

0 4 6 0.5 3.115234 6.857670 1.0 2.426171 7.632106 1.5 1.889523 8.326886 2.0 1.471577 8.946865

25.4.3 Computer Algorithm for Solving Systems of ODEs

The computer code for solving a single ODE with Euler’s method (Fig. 25.7) can be easily extended to systems of equations. The modifi cations include:

1. Inputting the number of equations, n. 2. Inputting the initial values for each of the n dependent variables. 3. Modifying the algorithm so that it computes slopes for each of the dependent

variables. 4. Including additional equations to compute derivative values for each of the ODEs. 5. Including loops to compute a new value for each dependent variable.

Such an algorithm is outlined in Fig. 25.18 for the fourth-order RK method. Notice how similar it is in structure and organization to Fig. 25.7. Most of the differences relate to the fact that

1. There are n equations. 2. The added detail of the fourth-order RK method.

EXAMPLE 25.11 Solving Systems of ODEs with the Computer

Problem Statement. A computer program to implement the fourth-order RK method for systems can be easily developed based on Fig. 25.18. Such software makes it con- venient to compare different models of a physical system. For example, a linear model for a swinging pendulum is given by [recall Eq. (PT7.11)]

dy1 dx

5 y2   dy2 dx

5 216.1y1

where y1 and y2 5 angular displacement and velocity. A nonlinear model of the same system is [recall Eq. (PT7.9)]

dy3 dx

5 y4   dy4 dx

5 216.1 sin(y3)

where y3 and y4 5 angular displacement and velocity for the nonlinear case. Solve these systems for two cases: (a) a small initial displacement (y1 5 y3 5 0.1 radians; y2 5 y4 5 0) and (b) a large displacement (y1 5 y3 5 py4 5 0.785398 radians; y2 5 y4 5 0).

cha9792x_ch25_709-754.indd Page 742 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 742 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.4 SYSTEMS OF EQUATIONS 743

(a) Main or “Driver” Program

Assign values for n 5 number of equations yi 5 initial values of n dependent

variables xi 5 initial value independent

variable xf 5 final value independent variable dx 5 calculation step size xout 5 output interval

x 5 xi m 5 0 xpm 5 x DOFOR i 5 1, n ypi,m 5 yii yi 5 yii END DO DO xend 5 x 1 xout IF (xend . xf) THEN xend 5 xf h 5 dx CALL Integrator (x, y, n, h, xend) m 5 m 1 1 xpm 5 x DOFOR i 5 1, n ypi,m 5 yi END DO IF (x $ xf) EXIT END DO DISPLAY RESULTS END

(b) Routine to Take One Output Step

SUB Integrator (x, y, n, h, xend) DO IF (xend 2 x , h) THEN h 5 xend 2 x CALL RK4 (x, y, n, h) IF (x $ xend) EXIT END DO END SUB

(c) Fourth-Order RK Method for a System of ODEs

SUB RK4 (x, y, n, h) CALL Derivs (x, y, k1) DOFOR i 5 1, n ymi 5 yi 1 k1i * h / 2 END DO CALL Derivs (x 1 h / 2, ym, k2) DOFOR i 5 1, n ymi 5 yi 1 k2i * h / 2 END DO CALL Derivs (x 1 h / 2, ym, k3) DOFOR i 5 1, n yei 5 yi 1 k3i * h END DO CALL Derivs (x 1 h, ye, k4) DOFOR i 5 1, n slopei 5 (k1i 1 2*(k2i1k3i)1k4i)/6 yi 5 yi 1 slopei * h END DO x 5 x 1 h END SUB

(d ) Routine to Determine Derivatives

SUB Derivs (x, y, dy) dy1 5 ... dy2 5 ... END SUB

FIGURE 25.18 Pseudocode for the fourth-order RK method for systems.

cha9792x_ch25_709-754.indd Page 743 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 743 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

744 RUNGE-KUTTA METHODS

Solution.

(a) The calculated results for the linear and nonlinear models are almost identical (Fig. 25.19a). This is as expected because when the initial displacement is small, sin (u) > u.

(b) When the initial displacement is py4 5 0.785398, the solutions are much different and the difference is magnifi ed as time becomes larger and larger (Fig. 25.19b). This is expected because the assumption that sin (u) 5 u is poor when theta is large.

25.5 ADAPTIVE RUNGE-KUTTA METHODS

To this point, we have presented methods for solving ODEs that employ a constant step size. For a signifi cant number of problems, this can represent a serious limitation. For example, suppose that we are integrating an ODE with a solution of the type depicted in Fig. 25.20. For most of the range, the solution changes gradually. Such behavior sug- gests that a fairly large step size could be employed to obtain adequate results. However, for a localized region from x 5 1.75 to x 5 2.25, the solution undergoes an abrupt change. The practical consequence of dealing with such functions is that a very small step size would be required to accurately capture the impulsive behavior. If a constant step-size al- gorithm were employed, the smaller step size required for the region of abrupt change would have to be applied to the entire computation. As a consequence, a much smaller step size than necessary—and, therefore, many more calculations—would be wasted on the regions of gradual change.

4

2

0y

0 321 x

– 4

– 2

4

y1, y3

y2, y4

(a)

4

2

0y

0 2 31 x

– 4

– 2

4

y2 y4

y3

y1

(b)

FIGURE 25.19 Solutions obtained with a computer program for the fourth-order RK method. The plots represent solutions for both linear and nonlinear pendulums with (a) small and (b) large initial displacements.

cha9792x_ch25_709-754.indd Page 744 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 744 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.5 ADAPTIVE RUNGE-KUTTA METHODS 745

Algorithms that automatically adjust the step size can avoid such overkill and hence be of great advantage. Because they “adapt” to the solution’s trajectory, they are said to have adaptive step-size control. Implementation of such approaches requires that an es- timate of the local truncation error be obtained at each step. This error estimate can then serve as a basis for either lengthening or decreasing the step size. Before proceeding, we should mention that aside from solving ODEs, the methods described in this chapter can also be used to evaluate defi nite integrals. As mentioned previously in the introduction to Part Six, the evaluation of the integral

I 5 # b

a f(x) dx

is equivalent to solving the differential equation

dy

dx 5 f(x)

for y(b) given the initial condition y(a) 5 0. Thus, the following techniques can be em- ployed to effi ciently evaluate defi nite integrals involving functions that are generally smooth but exhibit regions of abrupt change. There are two primary approaches to incorporate adaptive step-size control into one- step methods. In the fi rst, the error is estimated as the difference between two predictions using the same-order RK method but with different step sizes. In the second, the local

FIGURE 25.20 An example of a solution of an ODE that exhibits an abrupt change. Automatic step-size adjustment has great advantages for such cases.

1

0 1 2 3

y

x

cha9792x_ch25_709-754.indd Page 745 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 745 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

746 RUNGE-KUTTA METHODS

truncation error is estimated as the difference between two predictions using different- order RK methods.

25.5.1 Adaptive RK or Step-Halving Method

Step halving (also called adaptive RK) involves taking each step twice, once as a full step and independently as two half steps. The difference in the two results represents an estimate of the local truncation error. If y1 designates the single-step prediction and y2 designates the prediction using the two half steps, the error D can be represented as

¢ 5 y2 2 y1 (25.43)

In addition to providing a criterion for step-size control, Eq. (25.43) can also be used to correct the y2 prediction. For the fourth-order RK version, the correction is

y2 d y2 1 ¢ 15

(25.44)

This estimate is fi fth-order accurate.

EXAMPLE 25.12 Adaptive Fourth-Order RK Method

Problem Statement. Use the adaptive fourth-order RK method to integrate y9 5 4e0.8x 2 0.5y from x 5 0 to 2 using h 5 2 and an initial condition of y(0) 5 2. This is the same differential equation that was solved previously in Example 25.5. Recall that the true solutions is y(2) 5 14.84392.

Solution. The single prediction with a step of h is computed as

y(2) 5 2 1 1

6 [3 1 2(6.40216 1 4.70108) 1 14.11105]2 5 15.10584

The two half-step predictions are

y(1) 5 2 1 1

6 [3 1 2(4.21730 1 3.91297) 1 5.945681]1 5 6.20104

and

y(2) 5 6.20104 1 1

6 [5.80164 1 2(8.72954 1 7.99756) 1 12.71283]1 5 14.86249

Therefore, the approximate error is

Ea 5 14.86249 2 15.10584

15 5 20.01622

which compares favorably with the true error of

Et 5 14.84392 2 14.86249 5 20.01857

The error estimate can also be used to correct the prediction

y(2) 5 14.86249 2 0.01622 5 14.84627

which has an Et 5 20.00235.

cha9792x_ch25_709-754.indd Page 746 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 746 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.5 ADAPTIVE RUNGE-KUTTA METHODS 747

25.5.2 Runge-Kutta Fehlberg

Aside from step halving as a strategy to adjust step size, an alternative approach for obtaining an error estimate involves computing two RK predictions of different order. The results can then be subtracted to obtain an estimate of the local truncation error. One shortcoming of this approach is that it greatly increases the computational overhead. For example, a fourth- and fi fth-order prediction amount to a total of 10 function evaluations per step. The Runge-Kutta Fehlberg or embedded RK method cleverly circumvents this problem by using a fi fth-order RK method that employs the function evaluations from the accompanying fourth-order RK method. Thus, the approach yields the error estimate on the basis of only six function evaluations! For the present case, we use the following fourth-order estimate

yi11 5 yi 1 a 37378 k1 1 250

621 k3 1

125

594 k4 1

512

1771 k6b h (25.45)

along with the fi fth-order formula:

yi11 5 yi 1 a 282527,648 k1 1 18,575

48,384 k3 1

13,525

55,296 k4 1

277

14,336 k5 1

1

4 k6b h (25.46)

where

k1 5 f(xi, yi)

k2 5 f axi 1 15 h, yi 1 1

5 k1hb

k3 5 f axi 1 310 h, yi 1 3

40 k1h 1

9

40 k2hb

k4 5 f axi 1 35 h, yi 1 3

10 k1h 2

9

10 k2h 1

6

5 k3hb

k5 5 f axi 1 h, yi 2 1154 k1h 1 5

2 k2h 2

70

27 k3h 1

35

27 k4hb

k6 5 f axi 1 78 h, yi 1 1631

55,296 k1h 1

175

512 k2h 1

575

13,824 k3h 1

44,275

110,592 k4h

1 253

4096 k5hb

Thus, the ODE can be solved with Eq. (25.46) and the error estimated as the difference of the fi fth- and fourth-order estimates. It should be noted that the particular coeffi cients used above were developed by Cash and Karp (1990). Therefore, it is sometimes called the Cash-Karp RK method.

EXAMPLE 25.13 Runge-Kutta Fehlberg Method

Problem Statement. Use the Cash-Karp version of the Runge-Kutta Fehlberg approach to perform the same calculation as in Example 25.12 from x 5 0 to 2 using h 5 2.

cha9792x_ch25_709-754.indd Page 747 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 747 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

748 RUNGE-KUTTA METHODS

Solution. The calculation of the k’s can be summarized in the following table:

x y f(x, y)

k1 0 2 3 k2 0.4 3.2 3.908511 k3 0.6 4.20883 4.359883 k4 1.2 7.228398 6.832587 k5 2 15.42765 12.09831 k6 1.75 12.17686 10.13237

These can then be used to compute the fourth-order prediction

y1 5 2 1 a 37378 3 1 250

621 4.359883 1

125

594 6.832587 1

512

1771 10.13237b 2 5 14.83192

along with a fi fth-order formula:

y1 5 2 1 a 282527,648 3 1 18,575

48,384 4.359883 1

13,525

55,296 6.832587

1 227

14,336 12.09831 1

1

4 10.13237b 2 5 14.83677

The error estimate is obtained by subtracting these two equations to give

Ea 5 14.83677 2 14.83192 5 0.004842

25.5.3 Step-Size Control

Now that we have developed ways to estimate the local truncation error, it can be used to adjust the step size. In general, the strategy is to increase the step size if the error is too small and decrease it if the error is too large. Press et al. (2007) have suggested the following criterion to accomplish this:

hnew 5 hpresent ` ¢new¢present ` a

(25.47)

where hpresent and hnew 5 the present and the new step sizes, respectively, Dpresent 5 the computed present accuracy, Dnew 5 the desired accuracy, and a 5 a constant power that is equal to 0.2 when the step size is increased (that is, when Dpresent # Dnew) and 0.25 when the step size is decreased (Dpresent . Dnew). The key parameter in Eq. (25.47) is obviously Dnew because it is your vehicle for specifying the desired accuracy. One way to do this would be to relate Dnew to a rela- tive error level. Although this works well when only positive values occur, it can cause problems for solutions that pass through zero. For example, you might be simulating an oscillating function that repeatedly passes through zero but is bounded by maximum absolute values. For such a case, you might want these maximum values to fi gure in the desired accuracy.

cha9792x_ch25_709-754.indd Page 748 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 748 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.5 ADAPTIVE RUNGE-KUTTA METHODS 749

A more general way to handle such cases is to determine Dnew as

¢new 5 eyscale where e 5 an overall tolerance level. Your choice of yscale will then determine how the error is scaled. For example, if yscale 5 y, the accuracy will be couched in terms of fractional relative errors. If you are dealing with a case where you desire constant errors relative to a prescribed maximum bound, set yscale equal to that bound. A trick suggested by Press et al. (2007) to obtain the constant relative errors except very near zero crossings is

yscale 5 Zy Z 1 ` h dydx ` This is the version we will use in our algorithm.

25.5.4 Computer Algorithm

Figures 25.21 and 25.22 outline pseudocode to implement the Cash-Karp version of the Runge-Kutta Fehlberg algorithm. This algorithm is patterned after a more detailed imple- mentation by Press et al. (2007) for systems of ODEs. Figure 25.21 implements a single step of the Cash-Karp routine (that is Eqs. 25.45 and 25.46). Figure 25.22 outlines a general driver program along with a subroutine that actually adapts the step size.

SUBROUTINE RKkc (y,dy,x,h,yout,yerr) PARAMETER (a250.2,a350.3,a450.6,a551.,a650.875, b2150.2,b3153.y40.,b3259.y40.,b4150.3,b42520.9, b4351.2,b515211.y54.,b5252.5,b535270.y27., b54535.y27.,b6151631.y55296.,b625175.y512., b635575.y13824.,b64544275.y110592.,b655253.y4096., c1537.y378.,c35250.y621.,c45125.y594., c65512.y1771.,dc15c122825.y27648., dc35c3218575.y48384.,dc45c4213525.y55296., dc552277.y14336.,dc65c620.25) ytemp5y1b21*h*dy CALL Derivs (x1a2*h,ytemp,k2) ytemp5y1h*(b31*dy1b32*k2) CALL Derivs(x1a3*h,ytemp,k3) ytemp5y1h*(b41*dy1b42*k21b43*k3) CALL Derivs(x1a4*h,ytemp,k4) ytemp5y1h*(b51*dy1b52*k21b53*k31b54*k4) CALL Derivs(x1a5*h,ytemp,k5) ytemp5y1h*(b61*dy1b62*k21b63*k31b64*k41b65*k5) CALL Derivs(x1a6*h,ytemp,k6) yout5y1h*(c1*dy1c3*k31c4*k41c6*k6) yerr5h*(dc1*dy1dc3*k31dc4*k41dc5*k51dc6*k6) END RKkc

FIGURE 25.21 Pseudocode for a single step of the Cash-Karp RK method.

cha9792x_ch25_709-754.indd Page 749 09/12/13 9:13 AM F-468 cha9792x_ch25_709-754.indd Page 749 09/12/13 9:13 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

750 RUNGE-KUTTA METHODS

EXAMPLE 25.14 Computer Application of an Adaptive Fourth-Order RK Scheme

Problem Statement. The adaptive RK method is well-suited for the following ordinary differential equation

dy

dx 1 0.6y 5 10e2(x22)

2y[2(0.075)2] (E25.14.1)

Notice for the initial condition, y(0) 5 0.5, the general solution is

y 5 0.5e20.6x (E25.14.2)

which is a smooth curve that gradually approaches zero as x increases. In contrast, the particular solution undergoes an abrupt transition in the vicinity of x 5 2 due to the nature of the forcing function (Fig. 25.23a). Use a standard fourth-order RK scheme to solve Eq. (E25.14.1) from x 5 0 to 4. Then employ the adaptive scheme described in this sec- tion to perform the same computation.

Solution. First, the classical fourth-order scheme is used to compute the solid curve in Fig. 25.23b. For this computation, a step size of 0.1 is used so that 4y(0.1) 5 40 applica- tions of the technique are made. Then, the calculation is repeated with a step size of 0.05 for a total of 80 applications. The major discrepancy between the two results occurs in the region from 1.8 to 2.0. The magnitude of the discrepancy is about 0.1 to 0.2 percent.

(a) Driver Program

INPUT xi, xf, yi maxstep5100 hi5.5; tiny 5 1. 3 10230

eps50.00005 print *, xi,yi x5xi y5yi h5hi istep50 DO IF (istep . maxstep AND x # xf) EXIT istep5istep11 CALL Derivs(x,y,dy) yscal5ABS(y)1ABS(h*dy)1tiny IF (x1h.xf) THEN h5xf2x CALL Adapt (x,y,dy,h,yscal,eps,hnxt) PRINT x,y h5hnxt END DO END

(b) Adaptive Step Routine

SUB Adapt (x,y,dy,htry,yscal,eps,hnxt) PARAMETER (safety50.9, econ51.89e24) h5htry DO CALL RKkc (y,dy,x,h,ytemp,yerr) emax5abs(yerr/yscal/eps) IF emax # 1 EXIT htemp5safety*h*emax

20.25

h5max(abs(htemp),0.25*abs(h)) xnew5x1h IF xnew5x THEN pause END DO IF emax . econ THEN hnxt5safety*emax

2.2 *h

ELSE hnxt54.*h END IF x5x1h y5ytemp END Adapt

FIGURE 25.22 Pseudocode for a (a) driver program and an (b) adaptive step routine to solve a single ODE.

cha9792x_ch25_709-754.indd Page 750 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 750 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

25.5 ADAPTIVE RUNGE-KUTTA METHODS 751

Next, the algorithm in Figs. 25.21 and 25.22 is developed into a computer program and used to solve the same problem. An initial step size of 0.5 and an e 5 0.00005 were chosen. The results were superimposed on Fig. 25.23b. Notice how large steps are taken in the regions of gradual change. Then, in the vicinity of x 5 2, the steps are decreased to accommodate the abrupt nature of the forcing function.

FIGURE 25.23 (a) A bell-shaped forcing function that induces an abrupt change in the solution of an ODE [Eq. (E25.14.1)]. (b) The solution. The points indicate the predictions of an adaptive step-size routine.

0

1

2

0 2 4 x

(b)

0

5

10

0 2 4 x

(a)

The utility of an adaptive integration scheme obviously depends on the nature of the functions being modeled. It is particularly advantageous for those solutions with long smooth stretches and short regions of abrupt change. In addition, it has utility in those situations where the correct step size is not known a priori. For these cases, an adaptive routine will “feel” its way through the solution while keeping the results within the desired tolerance. Thus, it will tiptoe through the regions of abrupt change and step out briskly when the variations become more gradual.

cha9792x_ch25_709-754.indd Page 751 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 751 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

752 RUNGE-KUTTA METHODS

PROBLEMS

25.1 Solve the following initial value problem over the interval from t 5 0 to 2 where y(0) 5 1. Display all your results on the same graph.

dy

dt 5 yt 2 2 1.1y

(a) Analytically. (b) Euler’s method with h 5 0.5 and 0.25. (c) Midpoint method with h 5 0.5. (d) Fourth-order RK method with h 5 0.5. 25.2 Solve the following problem over the interval from x 5 0 to 1 using a step size of 0.25 where y(0) 5 1. Display all your results on the same graph.

dy

dt 5 (1 1 4t)1y

(a) Analytically. (b) Euler’s method. (c) Heun’s method without iteration. (d) Ralston’s method. (e) Fourth-order RK method. 25.3 Use the (a) Euler and (b) Heun (without iteration) methods to solve

d 2y

dt 2 2 0.5t 1 y 5 0

where y(0) 5 2 and y9(0) 5 0. Solve from x 5 0 to 4 using h 5 0.1. Compare the methods by plotting the solutions. 25.4 Solve the following problem with the fourth-order RK method:

d 2y

dx 2 1 0.6

dy

dx 1 8y 5 0

where y(0) 5 4 and y9(0) 5 0. Solve from x 5 0 to 5 with h 5 0.5. Plot your results. 25.5 Solve from t 5 0 to 3 with h 5 0.1 using (a) Heun (without corrector) and (b) Ralston’s second-order RK method:

dy

dt 5 y sin3(t)  y(0) 5 1

25.6 Solve the following problem numerically from t 5 0 to 3:

dy

dt 5 22y 1 t2   y(0) 5 1

Use the third-order RK method with a step size of 0.5. 25.7 Use (a) Euler’s and (b) the fourth-order RK method to solve

dy

dt 5 22y 1 5e2t

dz

dt 5 2

yz2

2

over the range t 5 0 to 0.4 using a step size of 0.1 with y(0) 5 2 and z(0) 5 4. 25.8 Compute the fi rst step of Example 25.14 using the adaptive fourth-order RK method with h 5 0.5. Verify whether step-size adjustment is in order. 25.9 If e 5 0.001, determine whether step size adjustment is re- quired for Example 25.12. 25.10 Use the RK-Fehlberg approach to perform the same calcula- tion as in Example 25.12 from x 5 0 to 1 with h 5 1. 25.11 Write a computer program based on Fig. 25.7. Among other things, place documentation statements throughout the program to identify what each section is intended to accomplish. 25.12 Test the program you developed in Prob. 25.11 by duplicat- ing the computations from Examples 25.1 and 25.4. 25.13 Develop a user-friendly program for the Heun method with an iterative corrector. Test the program by duplicating the results in Table 25.2. 25.14 Develop a user-friendly computer program for the classical fourth-order RK method. Test the program by duplicating Exam- ple 25.7. 25.15 Develop a user-friendly computer program for systems of equations using the fourth-order RK method. Use this program to duplicate the computation in Example 25.10. 25.16 The motion of a damped spring-mass system (Fig. P25.16) is described by the following ordinary differential equation:

m d 2x

dt2 1 c

dx

dt 1 kx 5 0

where x 5 displacement from equilibrium position (m), t 5 time (s), m 5 20-kg mass, and c 5 the damping coeffi cient (N ? s/m). The damping coeffi cient c takes on three values of 5 (under- damped), 40 (critically damped), and 200 (overdamped). The spring constant k 5 20 N/m. The initial velocity is zero, and the initial displacement x 5 1 m. Solve this equation using a numerical method over the time period 0 # t # 15 s. Plot the displacement versus time for each of the three values of the damping coeffi cient on the same curve.

FIGURE P25.16

k

c

x

m

cha9792x_ch25_709-754.indd Page 752 29/10/13 9:49 PM F-468 cha9792x_ch25_709-754.indd Page 752 29/10/13 9:49 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 753

25.21 The logistic model is used to simulate population as in

dp

dt 5 kgm(1 2 pypmax)p

where p 5 population, kgm 5 the maximum growth rate under un- limited conditions, and pmax 5 the carrying capacity. Simulate the world’s population from 1950 to 2000 using one of the numerical methods described in this chapter. Employ the following initial conditions and parameter values for your simulation: p0 (in 1950) 5 2555 million people, kgm 5 0.026/yr, and pmax 5 12,000 million people. Have the function generate output corresponding to the dates for the following measured population data. Develop a plot of your simulation along with these data.

t 1950 1960 1970 1980 1990 2000

p 2555 3040 3708 4454 5276 6079

25.22 Suppose that a projectile is launched upward from the earth’s surface. Assume that the only force acting on the object is the downward force of gravity. Under these conditions, a force balance can be used to derive,

dy

dt 5 2g(0)

R2

(R 1 x)2

where y 5 upward velocity (m/s), t 5 time (s), x 5 altitude (m) measured upwards from the earth’s surface, g(0) 5 the gravita- tional acceleration at the earth’s surface (> 9.81 m/s2), and R 5 the earth’s radius (> 6.37 3 106 m). Recognizing that dx/dt 5 y, use Euler’s method to determine the maximum height that would be obtained if y(t 5 0) 5 1500 m/s. 25.23 The following function exhibits both fl at and steep regions over a relatively short x region:

f (x) 5 1

(x 2 0.3)2 1 0.01 1

1

(x 2 0.9)2 1 0.04 2 6

Determine the value of the defi nite integral of this function between x 5 0 and 1 using an adaptive RK method.

25.17 If water is drained from a vertical cylindrical tank by open- ing a valve at the base, the water will fl ow fast when the tank is full and slow down as it continues to drain. As it turns out, the rate at which the water level drops is:

dy

dt 5 2k1y

where k is a constant depending on the shape of the hole and the cross-sectional area of the tank and drain hole. The depth of the water y is measured in meters and the time t in minutes. If k 5 0.06, determine how long it takes the tank to drain if the fl uid level is initially 3 m. Solve by applying Euler’s equation and writing a computer program or using Excel. Use a step of 0.5 minutes. 25.18 The following is an initial value, second-order differential equation:

d 2x

dt 2 1 (5x)

dx

dt 1 (x 1 7) sin(vt) 5 0

where

dx

dt (0) 5 1.5 and x(0) 5 6

Note that v 5 1. Decompose the equation into two fi rst-order dif- ferential equations. After the decomposition, solve the system from t 5 0 to 15 and plot the results. 25.19 Assuming that drag is proportional to the square of velocity, we can model the velocity of a falling object like a parachutist with the following differential equation:

dy

dt 5 g 2

cd m

y2

where y is velocity (m/s), t 5 time (s), g is the acceleration due to gravity (9.81 m/s2), cd 5 a second-order drag coeffi cient (kg/m), and m 5 mass (kg). Solve for the velocity and distance fallen by a 90-kg object with a drag coeffi cient of 0.225 kg/m. If the initial height is 1 km, determine when it hits the ground. Obtain your solu- tion with (a) Euler’s method and (b) the fourth-order RK method. 25.20 A spherical tank has a circular orifi ce in its bottom through which the liquid fl ows out (Fig. P25.20). The fl ow rate through the hole can be estimated as

Qout 5 CA12gH

where Qout 5 outfl ow (m 3/s), C 5 an empirically-derived coeffi -

cient, A 5 the area of the orifi ce (m2), g 5 the gravitational con- stant (5 9.81 m/s2), and H 5 the depth of liquid in the tank. Use one of the numerical methods described in this chapter to determine how long it will take for the water to fl ow out of a 3-m-diameter tank with an initial height of 2.75 m. Note that the orifi ce has a di- ameter of 3 cm and C 5 0.55.

FIGURE P25.20 A spherical tank.

H

r

cha9792x_ch25_709-754.indd Page 753 29/10/13 9:50 PM F-468 cha9792x_ch25_709-754.indd Page 753 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

754 RUNGE-KUTTA METHODS

from its equilibrium position (m), and g 5 gravitational acceleration (9.81 m/s2). Solve these equations for the positions and velocities of the three jumpers given the initial conditions that all positions and velocities are zero at t 5 0. Use the following parameters for your calculations: m1 5 60 kg, m2 5 70 kg, m3 5 80 kg, k1 5 k3 5 50, and k2 5 100 (N/m).

25.24 Given the initial conditions, y(0) 5 1 and y9(0) 5 0, solve the following initial-value problem from t 5 0 to 4:

d 2y

dt 2 1 4y 5 0

Obtain your solutions with (a) Euler’s method and (b) the fourth- order RK method. In both cases, use a step size of 0.125. Plot both solutions on the same graph along with the exact solution y 5 cos 2t. 25.25 Use the following differential equations to compute the velocity and position of a soccer ball that is kicked straight up in the air with an initial velocity of 40 m/s:

dy

dt 5 y

dv

dt 5 2g 2

cd m

y Zy Z

where y 5 upward distance (m), t 5 time (s), y 5 upward velocity (m/s), g 5 gravitational constant (5 9.81 m/s2), cd 5 drag coeffi - cient (kg/m), and m 5 mass (kg). Note that the drag coeffi cient is related to more fundamental parameters by

cd 5 1

2 rACd

where r 5 air density (kg/m3), A 5 area (m2), and Cd 5 the di- mensionless drag coeffi cient. Use the following parameter values for your calculation: d 5 22 cm, m 5 0.4 kg, r 5 1.3 kg/m3, and Cd 5 0.52. 25.26 Three linked bungee jumpers are depicted in Fig. P25.26. If the bungee cords are idealized as linear springs (i.e., governed by Hooke’s law), the following differential equations based on force balances can be developed

m1 d 2x1 dt 2

5 m1g 1 k2(x2 2 x1) 2 k1x1

m2 d 2x2 dt 2

5 m2g 1 k3(x3 2 x2) 1 k2(x1 2 x2)

m3 d 2x3 dt 2

5 m3g 1 k3(x2 2 x3)

where mi 5 the mass of jumper i (kg), kj 5 the spring constant for cord j (N/m), xi 5 the displacement of jumper i measured downward

FIGURE P25.26 Three individuals connected by bungee cords.

x1 = 0

(a) Unstretched (b) Stretched

x2 = 0

x3 = 0

cha9792x_ch25_709-754.indd Page 754 29/10/13 9:50 PM F-468 cha9792x_ch25_709-754.indd Page 754 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26 C H A P T E R 26

755

Stiffness and Multistep Methods

This chapter covers two areas. First, we describe stiff ODEs. These are both indi- vidual and systems of ODEs that have both fast and slow components to their solution. We introduce the idea of an implicit solution technique as one commonly used remedy for this problem. Then we discuss multistep methods. These algorithms retain informa- tion of previous steps to more effectively capture the trajectory of the solution. They also yield the truncation error estimates that can be used to implement adaptive step- size control.

26.1 STIFFNESS

Stiffness is a special problem that can arise in the solution of ordinary differential equa- tions. A stiff system is one involving rapidly changing components together with slowly changing ones. In many cases, the rapidly varying components are ephemeral transients that die away quickly, after which the solution becomes dominated by the slowly varying components. Although the transient phenomena exist for only a short part of the integra- tion interval, they can dictate the time step for the entire solution. Both individual and systems of ODEs can be stiff. An example of a single stiff ODE is

dy

dt 5 21000y 1 3000 2 2000e2t (26.1)

If y(0) 5 0, the analytical solution can be developed as

y 5 3 2 0.998e21000t 2 2.002e2t (26.2)

As in Fig. 26.1, the solution is initially dominated by the fast exponential term (e21000t). After a short period (t , 0.005), this transient dies out and the solution becomes dictated by the slow exponential (e2t). Insight into the step size required for stability of such a solution can be gained by examining the homogeneous part of Eq. (26.1),

dy

dt 5 2ay (26.3)

cha9792x_ch26_755-780.indd Page 755 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 755 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

756 STIFFNESS AND MULTISTEP METHODS

If y(0) 5 y0, calculus can be used to determine the solution as

y 5 y0e 2at

Thus, the solution starts at y0 and asymptotically approaches zero. Euler’s method can be used to solve the same problem numerically:

yi11 5 yi 1 dyi dt

h

Substituting Eq. (26.3) gives

yi11 5 yi 2 ayih

or

yi11 5 yi(1 2 ah) (26.4)

The stability of this formula clearly depends on the step size h. That is, 01 2 ah 0 must be less than 1. Thus, if h . 2ya, 0 yi 0 n q as i n q. For the fast transient part of Eq. (26.2), this criterion can be used to show that the step size to maintain stability must be , 2y1000 5 0.002. In addition, it should be noted that, whereas this criterion maintains stability (that is, a bounded solution), an even smaller step size would be required to obtain an accurate solution. Thus, although the transient occurs for only a small fraction of the integration interval, it controls the maximum allowable step size. Superfi cially, you might suppose that the adaptive step-size routines described at the end of the last chapter might offer a solution for this dilemma. You might think that they would use small steps during the rapid transients and large steps otherwise. However, this is not the case, because the stability requirement will still necessitate using very small steps throughout the entire solution.

FIGURE 26.1 Plot of a stiff solution of a single ODE. Although the solution appears to start at 1, there is actually a fast transient from y 5 0 to 1 that occurs in less than 0.005 time unit. This transient is perceptible only when the response is viewed on the fi ner timescale in the inset.

3

y

2

1

0 42 t0

1

0 0.020.010

cha9792x_ch26_755-780.indd Page 756 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 756 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.1 STIFFNESS 757

Rather than using explicit approaches, implicit methods offer an alternative remedy. Such representations are called implicit because the unknown appears on both sides of the equation. An implicit form of Euler’s method can be developed by evaluating the derivative at the future time,

yi11 5 yi 1 dyi11

dt h

This is called the backward, or implicit, Euler’s method. Substituting Eq. (26.3) yields

yi11 5 yi 2 ayi11 h

which can be solved for

yi11 5 yi

1 1 ah (26.5)

For this case, regardless of the size of the step, 0 yi 0 n 0 as i n q. Hence, the approach is called unconditionally stable.

EXAMPLE 26.1 Explicit and Implicit Euler

Problem Statement. Use both the explicit and implicit Euler methods to solve dy

dt 5 21000y 1 3000 2 2000e2t

where y(0) 5 0. (a) Use the explicit Euler with step sizes of 0.0005 and 0.0015 to solve for y between t 5 0 and 0.006. (b) Use the implicit Euler with a step size of 0.05 to solve for y between 0 and 0.4.

Solution.

(a) For this problem, the explicit Euler’s method is

yi11 5 yi 1 (21000yi 1 3000 2 2000e2ti)h

The result for h 5 0.0005 is displayed in Fig. 26.2a along with the analytical solu- tion. Although it exhibits some truncation error, the result captures the general shape of the analytical solution. In contrast, when the step size is increased to a value just below the stability limit (h 5 0.0015), the solution manifests oscillations. Using h  . 0.002 would result in a totally unstable solution, that is, it would go infi nite as the solution progressed.

(b) The implicit Euler’s method is

yi11 5 yi 1 (21000yi11 1 3000 2 2000e2ti11)h

Now because the ODE is linear, we can rearrange this equation so that yi11 is isolated on the left-hand side,

yi11 5 yi 1 3000h 2 2000he

2ti11

1 1 1000h

The result for h 5 0.05 is displayed in Fig. 26.2b along with the analytical solution. Notice that even though we have used a much bigger step size than the one that

cha9792x_ch26_755-780.indd Page 757 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 757 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

758 STIFFNESS AND MULTISTEP METHODS

induced instability for the explicit Euler, the numerical solution tracks nicely on the analytical result.

FIGURE 26.2 Solution of a “stiff” ODE with (a) the explicit and (b) implicit Euler methods.

1.5

y

1

0.5

0 0.0060.004

h = 0.0015

h = 0.0005 Exact

(a) t0 0.002

2

y

1

0 0.40.3

Exact

h = 0.05

(b) t0 0.20.1

Systems of ODEs can also be stiff. An example is

dy1 dt

5 25y1 1 3y2 (26.6a)

dy2 dt

5 100y1 2 301y2 (26.6b)

For the initial conditions y1(0) 5 52.29 and y2(0) 5 83.82, the exact solution is

y1 5 52.96e 23.9899t 2 0.67e2302.0101t (26.7a)

y2 5 17.83e 23.9899t 1 65.99e2302.0101t (26.7b)

Note that the exponents are negative and differ by about 2 orders of magnitude. As with the single equation, it is the large exponents that respond rapidly and are at the heart of the system’s stiffness.

cha9792x_ch26_755-780.indd Page 758 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 758 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 759

An implicit Euler’s method for systems can be formulated for the present example as

y1, i11 5 y1, i 1 (25y1, i11 1 3y2, i11)h (26.8a)

y2, i11 5 y2, i 1 (100y1, i11 2 301y2, i11)h (26.8b)

Collecting terms gives

(1 1 5h)y1, i11 2 3hy2, i11 5 y1, i (26.9a)

2100hy1, i11 1 (1 1 301h)y2, i11 5 y2, i (26.9b)

Thus, we can see that the problem consists of solving a set of simultaneous equations for each time step. For nonlinear ODEs, the solution becomes even more diffi cult since it involves solving a system of nonlinear simultaneous equations (recall Sec. 6.6). Thus, although stability is gained through implicit approaches, a price is paid in the form of added solu- tion complexity. The implicit Euler method is unconditionally stable and only fi rst-order accurate. It is also possible to develop in a similar manner a second-order accurate implicit trapezoi- dal rule integration scheme for stiff systems. It is usually desirable to have higher-order methods. The Adams-Moulton formulas described later in this chapter can also be used to devise higher-order implicit methods. However, the stability limits of such approaches are very stringent when applied to stiff systems. Gear (1971) developed a special series of implicit schemes that have much larger stability limits based on backward difference formulas. Extensive efforts have been made to develop software to effi ciently implement Gear’s methods. As a result, this is probably the most widely used method to solve stiff systems. In addition, Rosenbrock and others (see Press et al., 2007) have proposed implicit Runge-Kutta algorithms where the k terms appear implicitly. These methods have good stability characteristics and are quite suitable for solving systems of stiff ordinary differential equations.

26.2 MULTISTEP METHODS

The one-step methods described in the previous sections utilize information at a single point xi to predict a value of the dependent variable yi11 at a future point xi11 (Fig. 26.3a). Alternative approaches, called multistep methods (Fig. 26.3b), are based on the insight that, once the computation has begun, valuable information from previous points is at our command. The curvature of the lines connecting these previous values provides information regarding the trajectory of the solution. The multistep methods explored in this chapter exploit this information to solve ODEs. Before describing the higher-order versions, we will present a simple second-order method that serves to demonstrate the general characteristics of multistep approaches.

26.2.1 The Non-Self-Starting Heun Method

Recall that the Heun approach uses Euler’s method as a predictor [Eq. (25.15)]:

y0i11 5 yi 1 f(xi, yi)h (26.10)

cha9792x_ch26_755-780.indd Page 759 09/12/13 9:13 AM F-468 cha9792x_ch26_755-780.indd Page 759 09/12/13 9:13 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

760 STIFFNESS AND MULTISTEP METHODS

and the trapezoidal rule as a corrector [Eq. (25.16)]:

yi11 5 yi 1 f(xi, yi) 1 f(xi11, y

0 i11)

2 h (26.11)

Thus, the predictor and the corrector have local truncation errors of O(h2) and O(h3), respectively. This suggests that the predictor is the weak link in the method because it has the greatest error. This weakness is signifi cant because the effi ciency of the iterative corrector step depends on the accuracy of the initial prediction. Consequently, one way to improve Heun’s method is to develop a predictor that has a local error of O(h3). This can be accomplished by using Euler’s method and the slope at yi, and extra information from a previous point yi21, as in

y0i11 5 yi21 1 f(xi, yi)2h (26.12)

Notice that Eq. (26.12) attains O(h3) at the expense of employing a larger step size, 2h. In addition, note that Eq. (26.12) is not self-starting because it involves a previous value of the dependent variable yi 2 1. Such a value would not be available in a typical initial-value problem. Because of this fact, Eqs. (26.11) and (26.12) are called the non-self-starting Heun method. As depicted in Fig. 26.4, the derivative estimate in Eq. (26.12) is now located at the midpoint rather than at the beginning of the interval over which the prediction is made. As demonstrated subsequently, this centering improves the error of the predictor to O(h3). However, before proceeding to a formal derivation of the non-self-starting Heun, we will summarize the method and express it using a slightly modifi ed nomenclature:

Predictor: y0i11 5 y m i21 1 f(xi, ymi )2h (26.13)

Corrector: y ji11 5 y m i 1

f(xi, y mi ) 1 f(xi11, y j21i11 ) 2

h

(for j 5 1, 2, p , m) (26.14)

FIGURE 26.3 Graphical depiction of the fundamental difference between (a) one-step and (b) multistep methods for solving ODEs.

y

xi

(a) xxi + 1

y

xi

(b) xxi + 1xi – 1xi – 2

cha9792x_ch26_755-780.indd Page 760 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 760 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 761

where the superscripts have been added to denote that the corrector is applied iteratively from j 5 1 to m to obtain refi ned solutions. Note that ymi and y

m i21 are the fi nal results

of the corrector iterations at the previous time steps. The iterations are terminated at any time step on the basis of the stopping criterion

Zea Z 5 ` y j i11 2 y

j21 i11

y ji11 ` 100% (26.15)

When ea is less than a prespecifi ed error tolerance es, the iterations are terminated. At this point, j 5 m. The use of Eqs. (26.13) through (26.15) to solve an ODE is demonstrated in the following example.

EXAMPLE 26.2 Non-Self-Starting Heun Method

Problem Statement. Use the non-self-starting Heun method to perform the same com- putations as were performed previously in Example 25.5 using Heun’s method. That is,

FIGURE 26.4 A graphical depiction of the non-self-starting Heun method. (a) The midpoint method that is used as a predictor. (b) The trapezoidal rule that is employed as a corrector.

y

xxi+1

xi–1

xi

(a)

(b)

Slope = f (xi+1, yi+1) 0

y

xxi+1xi

Slope = f (xi, yi) + f (xi+1, yi+1)

2

0

cha9792x_ch26_755-780.indd Page 761 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 761 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

762 STIFFNESS AND MULTISTEP METHODS

integrate y9 5 4e0.8x 2 0.5y from x 5 0 to x 5 4 using a step size of 1.0. As with Example 25.5, the initial condition at x 5 0 is y 5 2. However, because we are now dealing with a multistep method, we require the additional information that y is equal to 20.3929953 at x 5 21.

Solution. The predictor [Eq. (26.13)] is used to extrapolate linearly from x 5 21 to x 5 1.

y01 5 20.3929953 1 [4e0.8(0) 2 0.5(2)] 2 5 5.607005

The corrector [Eq. (26.14)] is then used to compute the value:

y11 5 2 1 4e0.8(0) 2 0.5(2) 1 4e0.8(1) 2 0.5(5.607005)

2 1 5 6.549331

which represents a percent relative error of 25.73 percent (true value 5 6.194631). This error is somewhat smaller than the value of 28.18 percent incurred in the self-starting Heun. Now, Eq. (26.14) can be applied iteratively to improve the solution:

y21 5 2 1 3 1 4e0.8(1) 2 0.5(6.549331)

2 1 5 6.313749

which represents an et of 21.92%. An approximate estimate of the error can also be determined using Eq. (26.15):

0 ea 0 5 ` 6.313749 2 6.5493316.313749 ` 100% 5 3.7% Equation (26.14) can be applied iteratively until ea falls below a prespecifi ed value of es. As was the case with the Heun method (recall Example 25.5), the iterations converge on a value of 6.360865 (et 5 22.68%). However, because the initial predictor value is more accurate, the multistep method converges at a somewhat faster rate. For the second step, the predictor is

y02 5 2 1 [4e0.8(1) 2 0.5(6.360865)] 2 5 13.44346  et 5 9.43%

which is superior to the prediction of 12.08260 (et 5 18%) that was computed with the original Heun method. The fi rst corrector yields 15.76693 (et 5 6.8%), and subsequent iterations converge on the same result as was obtained with the self-starting Heun method: 15.30224 (et 5 23.1%). As with the previous step, the rate of convergence of the corrector is somewhat improved because of the better initial prediction.

Derivation and Error Analysis of Predictor-Corrector Formulas. We have just em- ployed graphical concepts to derive the non-self-starting Heun. We will now show how the same equations can be derived mathematically. This derivation is particularly interest- ing because it ties together ideas from curve fi tting, numerical integration, and ODEs. The exercise is also useful because it provides a simple procedure for developing higher- order multistep methods and estimating their errors. The derivation is based on solving the general ODE

dy

dx 5 f(x, y)

cha9792x_ch26_755-780.indd Page 762 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 762 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 763

This equation can be solved by multiplying both sides by dx and integrating between limits at i and i 1 1:

# yi11

yi dy 5 #

xi11

xi f(x, y) dx

The left side can be integrated and evaluated using [recall Eq. (25.21)]:

yi11 5 yi 1 # xi11

xi f(x, y) dx (26.16)

Equation (26.16) represents a solution to the ODE if the integral can be evaluated. That is, it provides a means to compute a new value of the dependent variable yi11 on the basis of a prior value yi and the differential equation. Numerical integration formulas such as those developed in Chap. 21 provide one way to make this evaluation. For example, the trapezoidal rule [Eq. (21.3)] can be used to evaluate the integral, as in

# xi11

xi f(x, y) dx 5

f(xi, yi) 1 f(xi11, yi11)

2 h (26.17)

where h 5 xi1 1 2 xi is the step size. Substituting Eq. (26.17) into Eq. (26.16) yields

yi11 5 yi 1 f(xi, yi) 1 f(xi11, yi11)

2 h

which is the corrector equation for the Heun method. Because this equation is based on the trapezoidal rule, the truncation error can be taken directly from Table 21.2,

Ec 5 2 1

12 h3y(3)(jc) 5 2

1

12 h3f –(jc) (26.18)

where the subscript c designates that this is the error of the corrector. A similar approach can be used to derive the predictor. For this case, the integration limits are from i 2 1 to i 1 1:

# yi11

yi21 dy 5 #

xi11

xi21 f(x, y) dx

which can be integrated and rearranged to yield

yi11 5 yi21 1 # xi11

xi21 f(x, y) dx (26.19)

Now, rather than using a closed formula from Table 21.2, the fi rst Newton-Cotes open integration formula (see Table 21.4) can be used to evaluate the integral, as in

# xi11

xi 2 1 f(x, y) dx 5 2h f(xi, yi) (26.20)

which is called the midpoint method. Substituting Eq. (26.20) into Eq. (26.19) yields

yi11 5 yi21 1 2h f(xi, yi)

cha9792x_ch26_755-780.indd Page 763 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 763 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

764 STIFFNESS AND MULTISTEP METHODS

which is the predictor for the non-self-starting Heun. As with the corrector, the local truncation error can be taken directly from Table 21.4:

Ep 5 1

3 h3y(3)(jp) 5

1

3 h3f –(jp) (26.21)

where the subscript p designates that this is the error of the predictor. Thus, the predictor and the corrector for the non-self-starting Heun method have truncation errors of the same order. Aside from upgrading the accuracy of the predic- tor, this fact has additional benefi ts related to error analysis, as elaborated in the next section.

Error Estimates. If the predictor and the corrector of a multistep method are of the same order, the local truncation error may be estimated during the course of a computa- tion. This is a tremendous advantage because it establishes a criterion for adjustment of the step size. The local truncation error for the predictor is estimated by Eq. (26.21). This error estimate can be combined with the estimate of yi1l from the predictor step to yield [recall our basic defi nition of Eq. (3.1)]

True value 5 y0i11 1 1

3 h3y(3)(jp) (26.22)

Using a similar approach, the error estimate for the corrector [Eq. (26.18)] can be com- bined with the corrector result yi1l to give

True value 5 y mi11 2 1

12 h3y(3)(jc) (26.23)

Equation (26.22) can be subtracted from Eq. (26.23) to yield

0 5 ymi11 2 y 0 i11 2

5

12 h3y(3)(j) (26.24)

where j is now between xi2l and xi1l. Now, dividing Eq. (26.24) by 5 and rearranging the result gives

y0i11 2 y m i11

5 5 2

1

12 h3y(3)(j) (26.25)

Notice that the right-hand sides of Eqs. (26.18) and (26.25) are identical, with the excep- tion of the argument of the third derivative. If the third derivative does not vary appre- ciably over the interval in question, we can assume that the right-hand sides are equal, and therefore, the left-hand sides should also be equivalent, as in

Ec 5 2 y0i11 2 y

m i11

5 (26.26)

Thus, we have arrived at a relationship that can be used to estimate the per-step truncation error on the basis of two quantities—the predictor (y0i11) and the corrector (ymi11)—that are routine by-products of the computation.

cha9792x_ch26_755-780.indd Page 764 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 764 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 765

EXAMPLE 26.3 Estimate of Per-Step Truncation Error

Problem Statement. Use Eq. (26.26) to estimate the per-step truncation error of Example 26.2. Note that the true values at x 5 1 and 2 are 6.194631 and 14.84392, respectively.

Solution. At xi1l 5 1, the predictor gives 5.607005 and the corrector yields 6.360865. These values can be substituted into Eq. (26.26) to give

Ec 5 2 6.360865 2 5.607005

5 5 20.1507722

which compares well with the exact error,

Et 5 6.194631 2 6.360865 5 20.1662341

At xi1l 5 2, the predictor gives 13.44346 and the corrector yields 15.30224, which can be used to compute

Ec 5 2 15.30224 2 13.44346

5 5 20.3717550

which also compares favorably with the exact error, Et 5 14.84392 2 15.30224 5 20.4583148.

The ease with which the error can be estimated using Eq. (26.26) provides a ratio- nal basis for step-size adjustment during the course of a computation. For example, if Eq. (26.26) indicates that the error is greater than an acceptable level, the step size could be decreased.

Modifi ers. Before discussing computer algorithms, we must note two other ways in which the non-self-starting Heun method can be made more accurate and effi cient. First, you should realize that besides providing a criterion for step-size adjustment, Eq. (26.26) represents a numerical estimate of the discrepancy between the fi nal corrected value at each step yi11 and the true value. Thus, it can be added directly to yi11 to refi ne the estimate further:

ymi11 d ymi11 2 ymi11 2 y

0 i11

5 (26.27)

Equation (26.27) is called a corrector modifi er. (The symbol m is read “is replaced by.”) The left-hand side is the modifi ed value of ymi11. A second improvement, one that relates more to program effi ciency, is a predictor modifi er, which is designed to adjust the predictor result so that it is closer to the fi nal convergent value of the corrector. This is advantageous because, as noted previously at the beginning of this section, the number of iterations of the corrector is highly dependent on the accuracy of the initial prediction. Consequently, if the prediction is modifi ed properly, we might reduce the number of iterations required to converge on the ultimate value of the corrector.

cha9792x_ch26_755-780.indd Page 765 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 765 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

766 STIFFNESS AND MULTISTEP METHODS

Such a modifi er can be derived simply by assuming that the third derivative is relatively constant from step to step. Therefore, using the result of the previous step at i, Eq. (26.25) can be solved for

h3y(3)(j) 5 2 12

5 (y0i 2 ymi ) (26.28)

which, assuming that y(3) (j) > y(3) (jp), can be substituted into Eq. (26.21) to give

Ep 5 4

5 (ymi 2 y0i ) (26.29)

which can then be used to modify the predictor result:

y0i11 d y0i11 1 4

5 (ymi 2 y0i ) (26.30)

EXAMPLE 26.4 Effect of Modifi ers on Predictor-Corrector Results

Problem Statement. Recompute Example 26.3 using both modifi ers.

Solution. As in Example 26.3, the initial predictor result is 5.607005. Because the predictor modifi er [Eq. (26.30)] requires values from a previous iteration, it cannot be employed to improve this initial result. However, Eq. (26.27) can be used to modify the corrected value of 6.360865 (et 5 22.684%), as in

ym1 5 6.360865 2 6.360865 2 5.607005

5 5 6.210093

which represents an et 5 20.25%. Thus, the error is reduced over an order of magnitude. For the next iteration, the predictor [Eq. (26.13)] is used to compute

y02 5 2 1 [4e0.8(0) 2 0.5(6.210093)] 2 5 13.59423  et 5 8.42%

which is about half the error of the predictor for the second iteration of Example 26.3, which was et 5 18.6%. This improvement occurs because we are using a superior estimate of y (6.210093 as opposed to 6.360865) in the predictor. In other words, the propagated and global errors are reduced by the inclusion of the corrector modifier. Now because we have information from the prior iteration, Eq. (26.30) can be em- ployed to modify the predictor, as in

y02 5 13.59423 1 4

5 (6.360865 2 5.607005) 5 14.19732  et 5 24.36%

which, again, halves the error. This modifi cation has no effect on the fi nal outcome of the subsequent corrector step. Regardless of whether the unmodifi ed or modifi ed predictors are used, the correc- tor will ultimately converge on the same answer. However, because the rate or effi ciency of convergence depends on the accuracy of the initial prediction, the modifi cation can reduce the number of iterations required for convergence.

cha9792x_ch26_755-780.indd Page 766 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 766 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 767

Implementing the corrector yields a result of 15.21178 (et 5 22.48%), which rep- resents an improvement over Example 26.3 because of the reduction of global error. Finally, this result can be modifi ed using Eq. (26.27):

ym2 5 15.21178 2 15.21178 2 13.59423

5 5 14.88827  et 5 20.30%

Again, the error has been reduced an order of magnitude.

As in the previous example, the addition of the modifi ers increases both the effi - ciency and accuracy of multistep methods. In particular, the corrector modifi er effectively increases the order of the technique. Thus, the non-self-starting Heun with modifi ers is third order rather than second order as is the case for the unmodifi ed version. However, it should be noted that there are situations where the corrector modifi er will affect the stability of the corrector iteration process. As a consequence, the modifi er is not included in the algorithm for the non-self-starting Heun delineated in Fig. 26.5. Nevertheless, the corrector modifi er can still have utility for step-size control, as discussed next.

FIGURE 26.5 The sequence of formulas used to implement the non-self-starting Heun method. Note that the corrector error estimates can be used to modify the corrector. However, because this can affect the corrector’s stability, the modifi er is not included in this algorithm. The corrector error estimate is included because of its utility for step-size adjustment.

Predictor: y0i11 5 yim21 1 f (xi, yim)2h

(Save result as y0i11,u 5 y0i11 where the subscript u designates that the variable is unmodifi ed.)

Predictor Modifi er:

y0i11 d y0i11,u 1 4 5

(y mi,u 2 y0i,u)

Corrector:

y ji11 5 y mi 1 f (xi, y mi ) 1 f (xi11, y j21i11)

2 h (for j 5 1 to maximum iterations m)

Error Check:

Zea Z 5 ` y j i11 2 y j21i11

y ji11 ` 100%

(If |ea| . error criterion, set j 5 j 11 and repeat corrector; if ea # error criterion, save result as yim11,u 5 yim11.)

Corrector Error Estimate:

Ec 5 2 1 5

(y mi11,u 2 y 0i11,u)

(If computation is to continue, set i 5 i 1 1 and return to predictor.)

cha9792x_ch26_755-780.indd Page 767 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 767 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

768 STIFFNESS AND MULTISTEP METHODS

26.2.2 Step-Size Control and Computer Programs

Constant Step Size. It is relatively simple to develop a constant step-size version of the non-self-starting Heun method. About the only complication is that a one-step method is required to generate the extra point to start the computation. Additionally, because a constant step size is employed, a value for h must be chosen prior to the computation. In general, experience indicates that an optimal step size should be small enough to ensure convergence within two iterations of the corrector (Hull and Creemer, 1963). In addition, it must be small enough to yield a suffi ciently small trunca- tion error. At the same time, the step size should be as large as possible to minimize run-time cost and round-off error. As with other methods for ODEs, the only practical way to assess the magnitude of the global error is to compare the results for the same problem but with a halved step size.

Variable Step Size. Two criteria are typically used to decide whether a change in step size is warranted. First, if Eq. (26.26) is greater than some prespecifi ed error criterion, the step size is decreased. Second, the step size is chosen so that the convergence criterion of the corrector is satisfi ed in two iterations. This criterion is intended to account for the trade-off between the rate of convergence and the total number of steps in the calculation. For smaller values of h, convergence will be more rapid but more steps are required. For larger h, convergence is slower but fewer steps result. Experience (Hull and Creemer, 1963) suggests that the total steps will be minimized if h is chosen so that the corrector converges within two iterations. Therefore, if over two iterations are required, the step size is decreased, and if less than two iterations are required, the step size is increased. Although the above strategy specifi es when step size modifi cations are in order, it does not indicate how they should be changed. This is a critical question because mul- tistep methods by defi nition require several points to compute a new point. Once the step size is changed, a new set of points must be determined. One approach is to restart the computation and use the one-step method to generate a new set of starting points. A more effi cient strategy that makes use of existing information is to increase and decrease by doubling and halving the step size. As depicted in Fig. 26.6b, if a suffi cient number of previous values have been generated, increasing the step size by doubling is a relatively straightforward task (Fig. 26.6c). All that is necessary is to keep track of subscripts so that old values of x and y become the appropriate new values. Halving the step size is somewhat more complicated because some of the new values will be unavail- able (Fig. 26.6a). However, interpolating polynomials of the type developed in Chap. 18 can be used to determine these intermediate values. In any event, the decision to incorporate step-size control represents a trade-off between initial investment in program complexity versus the long-term return because of increased effi ciency. Obviously, the magnitude and importance of the problem itself will have a strong bearing on this trade-off. Fortunately, several software packages and libraries have multistep routines that you can use to obtain solutions without having to program them from scratch. We will mention some of these when we review packages and libraries at the end of Chap. 27.

26.2.3 Integration Formulas

The non-self-starting Heun method is characteristic of most multistep methods. It em- ploys an open integration formula (the midpoint method) to make an initial estimate.

cha9792x_ch26_755-780.indd Page 768 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 768 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 769

This predictor step requires a previous data point. Then, a closed integration formula (the trapezoidal rule) is applied iteratively to improve the solution. It should be obvious that a strategy for improving multistep methods would be to use higher-order integration formulas as predictors and correctors. For example, the higher- order Newton-Cotes formulas developed in Chap. 21 could be used for this purpose. Before describing these higher-order methods, we will review the most common inte- gration formulas upon which they are based. As mentioned above, the fi rst of these are the Newton-Cotes formulas. However, there is a second class called the Adams formulas that we will also review and that are often preferred. As depicted in Fig. 26.7, the fundamental difference between the Newton-Cotes and Adams formulas relates to the manner in which the integral is applied to obtain the solution. As depicted in Fig. 26.7a, the Newton-Cotes formulas estimate the integral over an interval spanning several points. This integral is then used to project from the beginning of the interval to the end. In contrast, the Adams for- mulas (Fig. 26.7b) use a set of points from an interval to estimate the integral solely for the last segment in the interval. This integral is then used to project across this last segment.

FIGURE 26.6 A plot indicating how a halving-doubling strategy allows the use of (b) previously calculated val- ues for a third-order multistep method. (a) Halving; (c) doubling.

y

x

Interpolation

(a) y

x

(b) y

x

(c)

cha9792x_ch26_755-780.indd Page 769 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 769 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

770 STIFFNESS AND MULTISTEP METHODS

Newton-Cotes Formulas. Some of the most common formulas for solving ordinary differential equations are based on fi tting an nth-degree interpolating polynomial to n 1 1 known values of y and then using this equation to compute the integral. As discussed previously in Chap. 21, the Newton-Cotes integration formulas are based on such an approach. These formulas are of two types: open and closed forms.

Open Formulas. For n equally spaced data points, the open formulas can be expressed in the form of a solution of an ODE, as was done previously for Eq. (26.19). The general equation for this purpose is

yi11 5 yi2n 1 # xi11

xi2n fn(x) dx (26.31)

y

xi + 1 xxixi – 1

(a) xi – 2

yi + 1 = yi – 2 + xi + 1

xi – 2 � f (x, y) dx

y

xi + 1 xxixi – 1

(b) xi – 2

yi + 1 = yi + xi + 1

xi � f (x, y) dx

FIGURE 26.7 Illustration of the fundamental difference between the Newton-Cotes and Adams integration for- mulas. (a) The Newton-Cotes formulas use a series of points to obtain an integral estimate over a number of segments. The estimate is then used to project across the entire range. (b) The Adams formulas use a series of points to obtain an integral estimate for a single segment. The estimate is then used to project across the segment.

cha9792x_ch26_755-780.indd Page 770 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 770 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 771

where fn(x) is an nth-order interpolating polynomial. The evaluation of the integral em- ploys the nth-order Newton-Cotes open integration formula (Table 21.4). For example, if n 5 1,

yi11 5 yi21 1 2h fi (26.32)

where fi is an abbreviation for f(xi, yi)—that is, the differential equation evaluated at xi and yi. Equation (26.32) is referred to as the midpoint method and was used previously as the predictor in the non-self-starting Heun method. For n 5 2,

yi11 5 yi22 1 3h

2 ( fi 1 fi21)

and for n 5 3,

yi11 5 yi23 1 4h

3 (2 fi 2 fi21 1 2 fi22) (26.33)

Equation (26.33) is depicted graphically in Fig. 26.8a.

Closed Formulas. The closed form can be expressed generally as

yi11 5 yi2n11 1 # xi11

xi2n11 fn(x) dx (26.34)

where the integral is approximated by an nth-order Newton-Cotes closed integration formula (Table 21.2). For example, for n 5 1,

yi11 5 yi 1 h

2 ( fi 1 fi11)

which is equivalent to the trapezoidal rule. For n 5 2,

yi11 5 yi21 1 h

3 ( fi21 1 4fi 1 fi11) (26.35)

which is equivalent to Simpson’s 1y3 rule. Equation (26.35) is depicted in Fig. 26.8b.

Adams Formulas. The other types of integration formulas that can be used to solve ODEs are the Adams formulas. Many popular computer algorithms for multistep solution of ODEs are based on these methods.

Open Formulas (Adams-Bashforth). The Adams formulas can be derived in a variety of ways. One technique is to write a forward Taylor series expansion around xi:

yi11 5 yi 1 fi h 1 f ¿i 2

h2 1 f –i 6

h3 1 p

which can also be written as

yi11 5 yi 1 h afi 1 h2 f ¿i 1 h2

3 f –i 1 pb (26.36)

cha9792x_ch26_755-780.indd Page 771 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 771 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

772 STIFFNESS AND MULTISTEP METHODS

Recall from Sec. 4.1.3 that a backward difference can be used to approximate the derivative:

f ¿i 5 fi 2 fi21

h 1

f –i 2

h 1 O(h2)

which can be substituted into Eq. (26.36),

yi11 5 yi 1 h e fi 1 h2 c fi 2 fi21

h 1

f –i 2

h 1 O(h2) d 1 h2 6

f –i 1 p f or, collecting terms,

yi11 5 yi 1 h a32 fi 2 1

2 fi21b 1 512 h3 f –i 1 O(h4) (26.37)

y

xi + 1 xxixi – 1

(a) xi – 2xi – 3

y

xi + 1 xxixi – 1

(b)

FIGURE 26.8 Graphical depiction of open and closed Newton-Cotes integration formulas. (a) The third open formula [Eq. (26.33)] and (b) Simpson’s 1/3 rule [Eq. (26.35)].

cha9792x_ch26_755-780.indd Page 772 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 772 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 773

This formula is called the second-order open Adams formula. Open Adams formulas are also referred to as Adams-Bashforth formulas. Consequently, Eq. (26.37) is sometimes called the second Adams-Bashforth formula. Higher-order Adams-Bashforth formulas can be developed by substituting higher- difference approximations into Eq. (26.36). The nth-order open Adams formula can be represented generally as

yi11 5 yi 1 h a n21

k50 bk fi2k 1 O(hn11) (26.38)

The coeffi cients bk are compiled in Table 26.1. The fourth-order version is depicted in Fig. 26.9a. Notice that the fi rst-order version is Euler’s method.

Closed Formulas (Adams-Moulton). A backward Taylor series around xi1l can be written as

yi 5 yi11 2 fi11h 1 f ¿i11 2

h2 2 f –i11 3

h3 1 p

Solving for yi1l yields

yi11 5 yi 1 h a fi11 2 h2 f ¿i11 1 h2

6 f –i11 1 pb (26.39)

A difference can be used to approximate the fi rst derivative:

f ¿i11 5 fi11 2 fi

h 1

f –i11 2

h 1 O(h2)

TABLE 26.1 Coeffi cients and truncation error for Adams-Bashforth predictors.

Local Truncation Order B0 B1 B2 B3 B4 B5 Error

1 1 1 2

h2f ¿(j)

2 3/2 21/2 5

12 h3f ¿¿(j)

3 23/12 216/12 5/12 9

24 h4f 132(j)

4 55/24 259/24 37/24 29/24 251 720

h5f 142(j)

5 1901/720 22774/720 2616/720 21274/720 251/720 475

1440 h6f 152(j)

6 4277/720 27923/720 9982/720 27298/720 2877/720 2475/720 19,087 60,480

h7f 162(j)

cha9792x_ch26_755-780.indd Page 773 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 773 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

774 STIFFNESS AND MULTISTEP METHODS

which can be substituted into Eq. (26.39), and collecting terms gives

yi11 5 yi 1 h a12 fi11 1 1

2 fib 2 112 h3f –i11 2 O(h4)

This formula is called the second-order closed Adams formula or the second Adams- Moulton formula. Also, notice that it is the trapezoidal rule. The nth-order closed Adams formula can be written generally as

yi11 5 yi 1 h a n21

k50 bk fi112k 1 O(hn11)

The coeffi cients bk are listed in Table 26.2. The fourth-order method is depicted in Fig. 26.9b.

FIGURE 26.9 Graphical depiction of open and closed Adams integration formulas. (a) The fourth Adams- Bashforth open formula and (b) the fourth Adams-Moulton closed formula.

y

xi + 1 xxixi – 1

(a) xi – 2xi – 3

y

xi + 1 xxixi – 1

(b) xi – 2

cha9792x_ch26_755-780.indd Page 774 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 774 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 775

26.2.4 Higher-Order Multistep Methods

Now that we have formally developed the Newton-Cotes and Adams integration formu- las, we can use them to derive higher-order multistep methods. As was the case with the non-self-starting Heun method, the integration formulas are applied in tandem as predictor- corrector methods. In addition, if the open and closed formulas have local truncation

Box 26.1 Derivation of General Relationships for Modifi ers

The relationship between the true value, the approximation, and the error of a predictor can be represented generally as

True value 5 y0i11 1 hp

dp hn11 y(n11)(jp) (B26.1.1)

where hp and dp 5 the numerator and denominator, respectively, of the constant of the truncation error for either an open Newton- Cotes (Table 21.4) or an Adams-Bashforth (Table 26.1) predictor, and n is the order. A similar relationship can be developed for the corrector:

True value 5 ymi11 2 hc

dc hn11 y(n11)(jc) (B26.1.2)

where hc and dc 5 the numerator and denominator, respectively, of the constant of the truncation error for either a closed Newton- Cotes (Table 21.2) or an Adams-Moulton (Table 26.2) corrector. As was done in the derivation of Eq. (26.24), Eq. (B26.1.1) can be subtracted from Eq. (B26.1.2) to yield

0 5 ymi11 2 y 0 i11 2

hc 1 hpdcydp dc

hn11 y(n11)(j) (B26.1.3)

Now, dividing the equation by hc 1 hpdcydp, multiplying the last term by dpydp, and rearranging provides an estimate of the local

truncation error of the corrector:

Ec > 2 hcdp

hcdp 1 hpdc (ymi11 2 y0i11) (B26.1.4)

For the predictor modifi er, Eq. (B26.1.3) can be solved at the previous step for

hny(n11)(j) 5 2 dcdp

hcdp 1 hpdc (y0i 2 ymi )

which can be substituted into the error term of Eq. (B26.1.1) to yield

Ep 5 hpdc

hcdp 1 hpdc (ymi 2 y0i ) (B26.1.5)

Equations (B26.1.4) and (B26.1.5) are general versions of modifi - ers that can be used to improve multistep algorithms. For example, Milne’s method has hp 5 14, dp 5 45, hc 5 1, dc 5 90. Substituting these values into Eqs. (B26.1.4) and (B26.1.5) yields Eqs. (26.43) and (26.42), respectively. Similar modifi ers can be developed for other pairs of open and closed formulas that have local truncation errors of the same order.

TABLE 26.2 Coeffi cients and truncation error for Adams-Moulton correctors.

Local Truncation Order B0 B1 B2 B3 B B5 Error

2 1/2 1/2 2 1

12 h3f –(j)

3 5/12 8/12 21/12 2 1

24 h4f  (3)(j)

4 9/24 19/24 25/24 1/24 2 19

720 h5f  (4)(j)

5 251/720 646/720 2264/720 106/720 219/720 2 27

1440 h6f  (5)(j)

6 475/1440 1427/1440 2798/1440 482/1440 2173/1440 27/1440 2 863

60,480 h7f  (6)(j)

cha9792x_ch26_755-780.indd Page 775 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 775 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

776 STIFFNESS AND MULTISTEP METHODS

errors of the same order, modifi ers of the type listed in Fig. 26.5 can be incorporated to improve accuracy and allow step-size control. Box 26.1 provides general equations for these modifi ers. In the following section, we present two of the most common higher- order multistep approaches: Milne’s method and the fourth-order Adams method.

Milne’s Method. Milne’s method is the most common multistep method based on Newton-Cotes integration formulas. It uses the three-point Newton-Cotes open formula as a predictor:

y0i11 5 y m i23 1

4h

3 (2 f mi 2 f mi21 1 2 f mi22) (26.40)

and the three-point Newton-Cotes closed formula (Simpson’s 1y3 rule) as a corrector:

y ji11 5 y m i21 1

h

3 ( f mi21 1 4 f mi 1 f

j21 i11) (26.41)

where j is an index representing the number of iterations of the modifi er. The predictor and corrector modifi ers for Milne’s method can be developed from the formulas in Box 26.1 and the error coeffi cients in Tables 21.2 and 21.4:

Ep 5 28

29 (ymi 2 y0i ) (26.42)

Ec > 2 1

29 (ymi11 2 y0i11) (26.43)

EXAMPLE 26.5 Milne’s Method

Problem Statement. Use Milne’s method to integrate y9 5 4e0.8x 2 0.5y from x 5 0 to x 5 4 using a step size of 1. The initial condition at x 5 0 is y 5 2. Because we are dealing with a multistep method, previous points are required. In an actual application, a one-step method such as a fourth-order RK would be used to compute the required points. For the present example, we will use the analytical solution [recall Eq. (E25.5.1) from Example 25.5] to compute exact values at xi23 5 23, xi22 5 22, and xi21 5 21 of yi23 5 24.547302, yi22 5 22.306160, and yi21 5 20.3929953, respectively.

Solution. The predictor [Eq. (26.40)] is used to calculate a value at x 5 1:

y01 5 24.54730 1 4(1)

3 [2(3) 2 1.99381 1 2(1.96067)] 5 6.02272  et 5 2.8%

The corrector [Eq. (26.41)] is then employed to compute

y11 5 20.3929953 1 1

3 [1.99381 1 4(3) 1 5.890802] 5 6.235210  et 5 20.66%

This result can be substituted back into Eq. (26.41) to iteratively correct the estimate. This process converges on a fi nal corrected value of 6.204855 (et 5 20.17%). This value is more accurate than the comparable estimate of 6.360865 (et 522.68%) obtained previously with the non-self-starting Heun method (Examples 26.2 through 26.4). The results for the remaining steps are y(2) 5 14.86031 (et 5 20.11%), y(3) 5 33.72426 (et 5 20.14%), and y(4) 5 75.43295 (et 5 20.12%).

cha9792x_ch26_755-780.indd Page 776 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 776 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

26.2 MULTISTEP METHODS 777

As in the previous example, Milne’s method usually yields results of high accuracy. However, there are certain cases where it performs poorly (see Ralston and Rabinowitz, 1978). Before elaborating on these cases, we will describe another higher-order multistep approach—the fourth-order Adams method.

Fourth-Order Adams Method. A popular multistep method based on the Adams integration formulas uses the fourth-order Adams-Bashforth formula (Table 26.1) as the predictor:

y0i11 5 y m i 1 h a5524 f mi 2

59

24 f mi21 1

37

24 f mi22 2

9

24 f mi23b (26.44)

and the fourth-order Adams-Moulton formula (Table 26.2) as the corrector:

y ji11 5 y m i 1 h a 924 f j21i11 1

19

24 f mi 2

5

24 f mi21 1

1

24 f mi22b (26.45)

The predictor and the corrector modifi ers for the fourth-order Adams method can be developed from the formulas in Box 26.1 and the error coeffi cients in Tables 26.1 and 26.2 as

Ep 5 251

270 (ymi 2 y0i ) (26.46)

Ec 5 2 19

270 (ymi11 2 y0i11) (26.47)

EXAMPLE 26.6 Fourth-Order Adams Method

Problem Statement. Use the fourth-order Adams method to solve the same problem as in Example 26.5.

Solution. The predictor [Eq. (26.44)] is used to compute a value at x 5 1.

y01 5 2 1 1 a5524 3 2 59

24 1.993814 1

37

24 1.960667 2

9

24 2.6365228b 5 6.007539

et 5 3.1%

which is comparable to but somewhat less accurate than the result using the Milne method. The corrector [Eq. (26.45)] is then employed to calculate

y11 5 2 1 1 a 924 5.898394 1 19

24 3 2

5

24 1.993814 1

1

24 1.960666b 5 6.253214

et 5 20.96%

which again is comparable to but less accurate than the result using Milne’s method. This result can be substituted back into Eq. (26.45) to iteratively correct the estimate. The process converges on a fi nal corrected value of 6.214424 (et 5 0.32%), which is an accurate result but again somewhat inferior to that obtained with the Milne method.

cha9792x_ch26_755-780.indd Page 777 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 777 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

778 STIFFNESS AND MULTISTEP METHODS

Stability of Multistep Methods. The superior accuracy of the Milne method exhibited in Examples 26.5 and 26.6 would be anticipated on the basis of the error terms for the predictors [Eqs. (26.42) and (26.46)] and the correctors [Eqs. (26.43) and (26.47)]. The coeffi cients for the Milne method, 14y45 and 1y90, are smaller than for the fourth-order Adams, 251y720 and 19y720. Additionally, the Milne method employs fewer function evaluations to attain these higher accuracies. At face value, these results might lead to the conclusion that the Milne method is superior and, therefore, preferable to the fourth-order Adams. Although this conclusion holds for many cases, there are instances where the Milne method performs unacceptably. Such behavior is exhibited in the following example.

EXAMPLE 26.7 Stability of Milne’s and Fourth-Order Adams Methods

Problem Statement. Employ Milne’s and the fourth-order Adams methods to solve

dy

dx 5 2y

with the initial condition that y 5 1 at x 5 0. Solve this equation from x 5 0 to x 5 10 using a step size of h 5 0.5. Note that the analytical solution is y 5 e2x.

Solution. The results, as summarized in Fig. 26.10, indicate problems with Milne’s method. Shortly after the onset of the computation, the errors begin to grow and oscillate

FIGURE 26.10 Graphical depiction of the instability of Milne’s method.

0.005

0 5 10 x

y

Milne’s method

True solution

cha9792x_ch26_755-780.indd Page 778 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 778 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 779

in sign. By x 5 10, the relative error has infl ated to 2831 percent and the predicted value itself has started to oscillate in sign. In contrast, the results for the Adams method would be much more acceptable. Although the error also grows, it would do so at a slow rate. Additionally, the discrepancies would not exhibit the wild swings in sign exhibited by the Milne method.

The unacceptable behavior manifested in the previous example by the Milne method is referred to as instability. Although it does not always occur, its possibility leads to the conclusion that Milne’s approach should be avoided. Thus, the fourth-order Adams method is normally preferred. The instability of Milne’s method is due to the corrector. Consequently, attempts have been made to rectify the shortcoming by developing stable correctors. One com- monly used alternative that employs this approach is Hamming’s method, which uses the Milne predictor and a stable corrector:

y ji11 5 9ymi 2 y

m i22 1 3h(y

j21 i11 1 2 f

m i 2 f

m i21)

8

which has a local truncation error:

Ec 5 1

40 h5y(4)(jc)

Hamming’s method also includes modifi ers of the form

Ep 5 9

121 (ymi 2 y0i )

Ec 5 2 112

121 (ymi11 2 y0i11)

The reader can obtain additional information on this and other multistep methods else- where (Hamming, 1973; Lapidus and Seinfi eld, 1971).

PROBLEMS

26.1 Given

dy

dx 5 2200,000y 1 200,000e2x 2e2x

(a) Estimate the step-size required to maintain stability using the explicit Euler method.

(b) If y(0) 5 0, use the implicit Euler to obtain a solution from t 5 0 to 2 using a step size of 0.1.

26.2 Given

dy

dt 5 30(cos t 2 y) 1 3 sin t

If y(0) 5 1, use the implicit Euler to obtain a solution from t 5 0 to 4 using a step size of 0.4. 26.3 Given

dx1 dt

5 1999x1 1 2999x2

dx2 dt

5 22000x1 2 3000x2

If x1(0) 5 x2(0) 5 1, obtain a solution from t 5 0 to 0.2 using a step size of 0.05 with the (a) explicit and (b) implicit Euler methods.

cha9792x_ch26_755-780.indd Page 779 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 779 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

780 STIFFNESS AND MULTISTEP METHODS

26.13 Consider the thin rod of length l moving in the x-y plane as shown in Fig. P26.13. The rod is fi xed with a pin on one end and a mass at the other. Note that g 5 9.81 m/s2 and l 5 0.5 m. This sys- tem can be solved using

u $

2 g

l u 5 0

Let u 5 0 and u # (0) 5 0.25 rad/s. Solve using any method studied

in this chapter. Plot the angle versus time and the angular velocity versus time. (Hint: Decompose the second-order ODE.) 26.14 Given the fi rst-order ODE

dx

dt 5 2700x 2 1000e2t

x(t 5 0) 5 4

Solve this stiff differential equation using a numerical method over the time period 0 # t # 5. Also solve analytically and plot the ana- lytic and numerical solution for both the fast transient and slow transition phase of the timescale. 26.15 The following second-order ODE is considered to be stiff

d 2y

dx2 5 21001

dy

dx 2 1000y

Solve this differential equation (a) analytically and (b) numerically for x 5 0 to 5. For (b) use an implicit approach with h 5 0.5. Note that the initial conditions are y(0) 5 1 and y9(0) 5 0. Display both results graphically. 26.16 Solve the following differential equation from t 5 0 to 1

dy

dt 5 210y

with the initial condition y(0) 5 1. Use the following techniques to obtain your solutions: (a) analytically, (b) the explicit Euler method, and (c) the implicit Euler method. For (b) and (c) use h 5 0.1 and 0.2. Plot your results.

26.4 Solve the following initial-value problem over the interval from t 5 2 to 3:

dy

dt 5 20.4y 1 e22t

Use the non-self-starting Heun method with a step size of 0.5 and initial conditions of y(l.5) 5 5.800007 and y(2.0) 5 4.762673. Iter- ate the corrector to es 5 0.1%. Compute the true percent relative errors et for your results based on the analytical solution. 26.5 Repeat Prob. 26.4, but use the fourth-order Adams method. [Note: y(0.5) 5 8.46909 and y(1.0) 5 7.037566.] Iterate the correc- tor to es 5 0.01%. 26.6 Solve the following initial-value problem from t 5 4 to 5:

dy

dt 5 2

2y

t

Use a step size of 0.5 and initial values of y(2.5) 5 0.48, y(3) 5 0.333333, y(3.5) 5 0.244898, and y(4) 5 0.1875. Obtain your solu- tions using the following techniques: (a) the non-self-starting Heun method (es 5 1%), and (b) the fourth-order Adams method (es 5 0.01%). [Note: The exact answers obtained analytically are y(4.5) 5 0.148148 and y(5) 5 0.12.] Compute the true percent relative errors et for your results. 26.7 Solve the following initial-value problem from x 5 0 to x 5 0.75:

dy

dx 5 yx2 2 y

Use the non-self-starting Heun method with a step size of 0.25. If y(0) 5 1, employ the fourth-order RK method with a step size of 0.25 to predict the starting value at y(0.25). 26.8 Solve the following initial-value problem from t 5 1.5 to t 5 2.5

dy

dt 5

22y

1 1 t

Use the fourth-order Adams method. Employ a step size of 0.5 and the fourth-order RK method to predict the start-up values if y(0) 5 2. 26.9 Develop a program for the implicit Euler method for a single linear ODE. Test it by duplicating Prob. 26.1b. 26.10 Develop a program for the implicit Euler method for a pair of linear ODEs. Test it by solving Eq. (26.6). 26.11 Develop a user-friendly program for the non-self-starting Heun method with a predictor modifi er. Employ a fourth-order RK method to compute starter values. Test the program by duplicating Example 26.4. 26.12 Use the program developed in Prob. 26.11 to solve Prob. 26.7.

FIGURE P26.13

m

l

cha9792x_ch26_755-780.indd Page 780 29/10/13 9:50 PM F-468 cha9792x_ch26_755-780.indd Page 780 29/10/13 9:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27 C H A P T E R 27

781

Boundary-Value and Eigenvalue Problems

Recall from our discussion at the beginning of Part Seven that an ordinary differential equation is accompanied by auxiliary conditions. These conditions are used to evaluate the constants of integration that result during the solution of the equation. For an nth-order equation, n conditions are required. If all the conditions are specifi ed at the same value of the independent variable, then we are dealing with an initial-value problem (Fig. 27.1a). To this point, the material in Part Seven has been devoted to this type of problem.

FIGURE 27.1 Initial-value versus boundary- value problems. (a) An initial- value problem where all the conditions are specifi ed at the same value of the independent variable. (b) A boundary-value problem where the conditions are specifi ed at different values of the independent variable.

y

y1

y2

t y2, 0

y1, 0

(a) 0

Initial conditions

Boundary condition

Boundary conditiony

yL

x

y0

(b) 0 L

= f1(t, y1, y2) dy1 dt

= f2(t, y1, y2) dy2 dt

where at t = 0, y1 = y1, 0 and y2 = y2, 0

= f (x, y) d2y

dx2

where at x = 0, y = y0 x = L, y = yL

cha9792x_ch27_781-810.indd Page 781 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 781 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

782 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

In contrast, there is another application for which the conditions are not known at a single point, but rather, are known at different values of the independent variable. Because these values are often specifi ed at the extreme points or boundaries of a system, they are customarily referred to as boundary-value problems (Fig. 27.1b). A variety of signifi cant engineering applications fall within this class. In this chapter, we discuss two general approaches for obtaining their solution: the shooting method and the fi nite- difference approach. Additionally, we present techniques to approach a special type of boundary-value problem: the determination of eigenvalues. Of course, eigenvalues also have many applications beyond those involving boundary-value problems.

27.1 GENERAL METHODS FOR BOUNDARY-VALUE PROBLEMS

The conservation of heat can be used to develop a heat balance for a long, thin rod (Fig. 27.2). If the rod is not insulated along its length and the system is at a steady state, the equation that results is

d 2T

dx2 1 h¿(Ta 2 T ) 5 0 (27.1)

where h9 is a heat transfer coeffi cient (m22) that parameterizes the rate of heat dissipation to the surrounding air and Ta is the temperature of the surrounding air (8C). To obtain a solution for Eq. (27.1), there must be appropriate boundary conditions. A simple case is where the temperatures at the ends of the rod are held at fi xed values. These can be expressed mathematically as

T(0) 5 T1 T(L) 5 T2

With these conditions, Eq. (27.1) can be solved analytically using calculus. For a 10-m rod with Ta 5 20, T1 5 40, T2 5 200, and h9 5 0.01, the solution is

T 5 73.4523e0.1x 2 53.4523e20.1x 1 20 (27.2)

In the following sections, the same problem will be solved using numerical approaches.

FIGURE 27.2 A noninsulated uniform rod positioned between two bodies of constant but different temperature. For this case T1 . T2 and T2 . Ta.

x = Lx = 0

T1 T2

Ta

Ta

cha9792x_ch27_781-810.indd Page 782 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 782 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.1 GENERAL METHODS FOR BOUNDARY-VALUE PROBLEMS 783

27.1.1 The Shooting Method

The shooting method is based on converting the boundary-value problem into an equiv- alent initial-value problem. A trial-and-error approach is then implemented to solve the initial-value version. The approach can be illustrated by an example.

EXAMPLE 27.1 The Shooting Method

Problem Statement. Use the shooting method to solve Eq. (27.1) for a 10-m rod with h9 5 0.01 m22, Ta 5 20, and the boundary conditions

T(0) 5 40  T(10) 5 200

Solution. Using the same approach as was employed to transform Eq. (PT7.2) into Eqs. (PT7.3) through (PT7.6), the second-order equation can be expressed as two fi rst- order ODEs:

dT

dx 5 z (E27.1.1)

dz

dx 5 h¿(T 2 Ta) (E27.1.2)

To solve these equations, we require an initial value for z. For the shooting method, we guess a value—say, z(0) 5 10. The solution is then obtained by integrating Eq. (E27.1.1) and (E27.1.2) simultaneously. For example, using a fourth-order RK method with a step size of 2, we obtain a value at the end of the interval of T(10) 5 168.3797 (Fig. 27.3a), which differs from the boundary condition of T(10) 5 200. Therefore, we make another guess, z(0) 5 20, and perform the computation again. This time, the result of T(10) 5 285.8980 is obtained (Fig. 27.3b). Now, because the original ODE is linear, the values

z(0) 5 10  T(10) 5 168.3797

and

z(0) 5 20  T(10) 5 285.8980

are linearly related. As such, they can be used to compute the value of z(0) that yields T(10) 5 200. A linear interpolation formula [recall Eq. (18.2)] can be employed for this purpose:

z(0) 5 10 1 20 2 10

285.8980 2 168.3797 (200 2 168.3797) 5 12.6907

This value can then be used to determine the correct solution, as depicted in Fig. 27.3c.

cha9792x_ch27_781-810.indd Page 783 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 783 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

784 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

Nonlinear Two-Point Problems. For nonlinear boundary-value problems, linear inter- polation or extrapolation through two solution points will not necessarily result in an accurate estimate of the required boundary condition to attain an exact solution. An al- ternative is to perform three applications of the shooting method and use a quadratic interpolating polynomial to estimate the proper boundary condition. However, it is un- likely that such an approach would yield the exact answer, and additional iterations would be necessary to obtain the solution. Another approach for a nonlinear problem involves recasting it as a roots problem. Recall that the general form of a roots problem is to fi nd the value of x that makes the

FIGURE 27.3 The shooting method: (a) the fi rst “shot,” (b) the second “shot,” and (c) the fi nal exact “hit.”

0 8 1064

(c)

(b)

20

100

200

0

100

200

(a) 0

100

200

cha9792x_ch27_781-810.indd Page 784 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 784 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.1 GENERAL METHODS FOR BOUNDARY-VALUE PROBLEMS 785

function f(x) 5 0. Now, let us use Example 27.1 to understand how the shooting method can be recast in this form. First, recognize that the solution of the pair of differential equations is also a “func- tion” in the sense that we guess a condition at the left-hand end of the rod, z0, and the integration yields a prediction of the temperature at the right-hand end, T10. Thus, we can think of the integration as

T10 5 f(z0)

That is, it represents a process whereby a guess of z0 yields a prediction of T10. Viewed in this way, we can see that what we desire is the value of z0 that yields a specifi c value of T10. If, as in the example, we desire T10 5 200, the problem can be posed as

200 5 f(z0)

By bringing the goal of 200 over to the right-hand side of the equation, we generate a new function, g(z0), that represents the difference between what we have, f(z0), and what we want, 200.

g(z0) 5 f(z0) 2 200

If we drive this new function to zero, we will obtain the solution. The next example illustrates the approach.

EXAMPLE 27.2 The Shooting Method for Nonlinear Problems

Problem Statement. Although it served our purposes for proving a simple boundary- value problem, our model for the rod in Eq. (27.1) was not very realistic. For one thing, such a rod would lose heat by mechanisms such as radiation that are nonlinear. Suppose that the following nonlinear ODE is used to simulate the temperature of the heated rod:

d 2T

dx2 1 h–(Ta 2 T)4 5 0

where h0 5 5 3 1028. Now, although it is still not a very good representation of heat transfer, this equation is straightforward enough to allow us to illustrate how the shoot- ing method can be used to solve a two-point nonlinear boundary-value problem. The remaining problem conditions are as specifi ed in Example 27.1.

Solution. The second-order equation can be expressed as two fi rst-order ODEs:

dT

dx 5 z

dz

dx 5 h–(T 2 Ta)4

Now, these equations can be integrated using any of the methods described in Chaps. 25 and 26. We used the constant step-size version of the fourth-order RK approach described in Chap. 25. We implemented this approach as an Excel macro function written in Visual

cha9792x_ch27_781-810.indd Page 785 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 785 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

786 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

BASIC. The function integrated the equations based on an initial guess for z(0) and returned the temperature at x 5 10. The difference between this value and the goal of 200 was then placed in a spreadsheet cell. The Excel Solver was then invoked to adjust the value of z(0) until the difference was driven to zero. The result is shown in Fig. 27.4 along with the original linear case. As might be expected, the nonlinear case is curved more than the linear model. This is due to the power of four term in the heat transfer relationship.

200

T, �C

100

0 z10

Nonlinear

Linear

50

FIGURE 27.4 The result of using the shooting method to solve a nonlinear problem.

The shooting method can become arduous for higher-order equations where the necessity to assume two or more conditions makes the approach somewhat more diffi cult. For these reasons, alternative methods are available, as described next.

27.1.2 Finite-Difference Methods

The most common alternatives to the shooting method are fi nite-difference approaches. In these techniques, fi nite divided differences are substituted for the derivatives in the original equation. Thus, a linear differential equation is transformed into a set of simul- taneous algebraic equations that can be solved using the methods from Part Three. For the case of Fig. 27.2, the fi nite-divided-difference approximation for the second derivative is (recall Fig. 23.3)

d 2T

dx2 5

Ti11 2 2Ti 1 Ti21 ¢x2

This approximation can be substituted into Eq. (27.1) to give

Ti11 2 2Ti 1 Ti21 ¢x2

2 h¿ (Ti 2 Ta) 5 0

cha9792x_ch27_781-810.indd Page 786 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 786 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.1 GENERAL METHODS FOR BOUNDARY-VALUE PROBLEMS 787

Collecting terms gives

2Ti21 1 (2 1 h¿ ¢x2)Ti 2 Ti11 5 h¿¢x2Ta (27.3)

This equation applies for each of the interior nodes of the rod. The fi rst and last interior nodes, Ti21 and Ti11, respectively, are specifi ed by the boundary conditions. Therefore, the resulting set of linear algebraic equations will be tridiagonal. As such, it can be solved with the effi cient algorithms that are available for such systems (Sec. 11.1).

EXAMPLE 27.3 Finite-Difference Approximation of Boundary-Value Problems

Problem Statement. Use the fi nite-difference approach to solve the same problem as in Example 27.1.

Solution. Employing the parameters in Example 27.1, we can write Eq. (27.3) for the rod from Fig. 27.2. Using four interior nodes with a segment length of Dx 5 2 m results in the following equations:D2.04 21 0 021 2.04 21 0

0 21 2.04 21

0 0 21 2.04

T dT1T2 T3 T4

t 5 d 40.80.8 0.8

200.8

t which can be solved for

{T}T 5 :65.9698 93.7785 124.5382 159.4795 ;

Table 27.1 provides a comparison between the analytical solution [Eq. (27.2)] and the numerical solutions obtained in Examples 27.1 and 27.3. Note that there are some discrepancies among the approaches. For both numerical methods, these errors can be mitigated by decreasing their respective step sizes. Although both techniques perform well for the present case, the fi nite-difference approach is preferred because of the ease with which it can be extended to more complex cases. The fi xed (or Dirichlet) boundary condition used in the previous example is but one of several types that are commonly employed in engineering and science. A common alterna- tive, called the Neumann boundary condition, is the case where the derivative is given.

TABLE 27.1 Comparison of the exact analytical solution with the shooting and fi nite- difference methods.

x True Shooting Method Finite Difference

0 40 40 40 2 65.9518 65.9520 65.9698 4 93.7478 93.7481 93.7785 6 124.5036 124.5039 124.5382 8 159.4534 159.4538 159.4795 10 200 200 200

cha9792x_ch27_781-810.indd Page 787 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 787 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

788 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

We can use the heated rod model to demonstrate how derivative boundary condition can be incorporated into the fi nite-difference approach,

0 5 d 2T

dx2 1 h¿ (Tq 2 T)

However, in contrast to our previous discussions, we will prescribe a derivative boundary condition at one end of the rod,

dT

dx (0) 5 T ¿a

T(L) 5 Tb

Thus, we have a derivative boundary condition at one end of the solution domain and a fi xed boundary condition at the other. As was done in Example 27.3, the rod is divided into a series of nodes and a fi nite- difference version of the differential equation (Eq. 27.3) is applied to each interior node. However, because its temperature is not specifi ed, the node at the left end must also be included. Writing Eq. (27.3) for this node gives

2T21 1 (2 1 h¿¢x2)T0 2 T1 5 h¿¢x2Tq (27.3a)

Notice that an imaginary node (21) lying to the left of the rod’s end is required for this equation. Although this exterior point might seem to represent a diffi culty, it actually serves as the vehicle for incorporating the derivative boundary condition into the prob- lem. This is done by representing the fi rst derivative in the x dimension at (0) by the centered difference

dT

dx 5

T1 2 T21 2¢x

which can be solved for

T21 5 T1 2 2¢x dT

dx

Now we have a formula for T21 that actually refl ects the impact of the derivative. It can be substituted into Eq. (27.3a) to give

(2 1 h¿¢x2)T0 2 2T1 5 h¿¢x2Tq 2 2¢x dT

dx (27.3b)

Consequently, we have incorporated the derivative into the balance. A common example of a derivative boundary condition is the situation where the end of the rod is insulated. In this case, the derivative is set to zero. This conclusion follows directly from Fourier’s law, which states that the heat fl ux is directly proportional to the temperature gradient. Thus, insulating a boundary means that the heat fl ux (and consequently the gradient) must be zero. Aside from the shooting and fi nite-difference methods, there are other techniques avail- able for solving boundary-value problems. Some of these will be described in Part Eight. These include steady-state (Chap. 29) and transient (Chap. 30) solution of two-dimensional

cha9792x_ch27_781-810.indd Page 788 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 788 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 789

boundary-value problems using fi nite differences and steady-state solutions of the one- dimensional problem with the fi nite-element approach (Chap. 31).

27.2 EIGENVALUE PROBLEMS

Eigenvalue, or characteristic-value, problems are a special class of boundary-value prob- lems that are common in engineering problem contexts involving vibrations, elasticity, and other oscillating systems. In addition, they are used in a wide variety of engineering contexts beyond boundary-value problems. Before describing numerical methods for solv- ing these problems, we will present some general background information. This includes discussion of both the mathematics and the engineering signifi cance of eigenvalues.

27.2.1 Mathematical Background

Part Three dealt with methods for solving sets of linear algebraic equations of the general form

[A]{X} 5 {B}

Such systems are called nonhomogeneous because of the presence of the vector {B} on the right-hand side of the equality. If the equations comprising such a system are linearly independent (that is, have a nonzero determinant), they will have a unique solution. In other words, there is one set of x values that will make the equations balance. In contrast, a homogeneous linear algebraic system has the general form

[A]{X} 5 0

Although nontrivial solutions (that is, solutions other than all x’s 5 0) of such systems are possible, they are generally not unique. Rather, the simultaneous equations establish relationships among the x’s that can be satisfi ed by various combinations of values. Eigenvalue problems associated with engineering are typically of the general form

(a11 2 l)x1 1  a12x2 1 p 1  a1n xn 5 0 a21x1 1 (a22 2 l)x2 1 p 1  a2n xn 5 0

. . . .

. . . .

. . . .

an1x1 1  an2x2 1 p 1 (ann 2 l)xn 5 0

where l is an unknown parameter called the eigenvalue, or characteristic value. A solution {X} for such a system is referred to as an eigenvector. The above set of equations may also be expressed concisely as

[[A] 2 l[I]]{X} 5 0 (27.4)

The solution of Eq. (27.4) hinges on determining l. One way to accomplish this is based on the fact that the determinant of the matrix [[A] 2 l[I]] must equal zero for nontrivial solutions to be possible. Expanding the determinant yields a polynomial in l. The roots of this polynomial are the solutions for the eigenvalues. An example of this approach will be provided in the next section.

cha9792x_ch27_781-810.indd Page 789 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 789 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

790 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

27.2.2 Physical Background

The mass-spring system in Fig. 27.5a is a simple context to illustrate how eigenvalues occur in physical problem settings. It also will help to illustrate some of the mathemat- ical concepts introduced in the previous section. To simplify the analysis, assume that each mass has no external or damping forces acting on it. In addition, assume that each spring has the same natural length l and the same spring constant k. Finally, assume that the displacement of each spring is measured relative to its own local coordinate system with an origin at the spring’s equilibrium position (Fig. 27.5a). Under these assumptions, Newton’s second law can be employed to develop a force balance for each mass (recall Sec. 12.4),

m1 d 2x1 dt2

5 2kx1 1 k(x2 2 x1)

and

m2 d 2x2 dt2

5 2k(x2 2 x1) 2 kx2

where xi is the displacement of mass i away from its equilibrium position (Fig. 27.5b). These equations can be expressed as

m1 d 2x1 dt2

2 k(22x1 1 x2) 5 0 (27.5a)

m2 d 2x2 dt2

2 k(x1 2 2x2) 5 0 (27.5b)

From vibration theory, it is known that solutions to Eq. (27.5) can take the form

xi 5 Ai sin(vt) (27.6)

x (a)

0

0

0 x1 0 x2

x (b)

m1 m2

m1 m2

FIGURE 27.5 Positioning the masses away from equilibrium creates forces in the springs that upon release lead to oscillations of the masses. The positions of the masses can be referenced to local coordinates with origins at their respective equilibrium positions.

cha9792x_ch27_781-810.indd Page 790 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 790 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 791

where Ai 5 the amplitude of the vibration of mass i and v 5 the frequency of the vibra- tion, which is equal to

v 5 2p

TP (27.7)

where Tp is the period. From Eq. (27.6) it follows that

x–i 5 2Ai v2 sin (vt) (27.8)

Equations (27.6) and (27.8) can be substituted into Eq. (27.5), which, after collection of terms, can be expressed as

a 2k m1

2 v 2b A1 2 km1 A2 5 0 (27.9a)

2 k

m2 A1 1 a 2km2 2 v2b A2 5 0 (27.9b)

Comparison of Eq. (27.9) with Eq. (27.4) indicates that at this point, the solution has been reduced to an eigenvalue problem.

EXAMPLE 27.4 Eigenvalues and Eigenvectors for a Mass-Spring System

Problem Statement. Evaluate the eigenvalues and the eigenvectors of Eq. (27.9) for the case where ml 5 m2 5 40 kg and k 5 200 N/m.

Solution. Substituting the parameter values into Eq. (27.9) yields

(10 2 v2) A1 2 5A2 5 0 25A1 1 (10 2 v2) A2 5 0

The determinant of this system is [recall Eq. (9.3)]

(v2)2 2 20v2 1 75 5 0

which can be solved by the quadratic formula for v2 5 15 and 5 s22. Therefore, the frequencies for the vibrations of the masses are v 5 3.873 s21 and 2.236 s21, respectively. These values can be used to determine the periods for the vibrations with Eq. (27.7). For the fi rst mode, Tp 5 1.62 s, and for the second, Tp 5 2.81 s. As stated in Sec. 27.2.1, a unique set of values cannot be obtained for the unknowns. However, their ratios can be specifi ed by substituting the eigenvalues back into the equa- tions. For example, for the fi rst mode (v2 5 15 s22), Al 5 2A2. For the second mode (v2 5 5 s22), A1 5 A2. This example provides valuable information regarding the behavior of the system in Fig. 27.5. Aside from its period, we know that if the system is vibrating in the fi rst mode, the amplitude of the second mass will be equal but of opposite sign to the amplitude of the fi rst. As in Fig. 27.6a, the masses vibrate apart and then together indefi nitely. In the second mode, the two masses have equal amplitudes at all times. Thus, as in Fig. 27.6b, they vibrate back and forth in unison. It should be noted that the confi gura- tion of the amplitudes provides guidance on how to set their initial values to attain pure

cha9792x_ch27_781-810.indd Page 791 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 791 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

792 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

motion in either of the two modes. Any other confi guration will lead to superposition of the modes (recall Chap. 19).

27.2.3 A Boundary-Value Problem

Now that you have been introduced to eigenvalues, we turn to the type of problem that is the subject of the present chapter: boundary-value problems for ordinary differential equations. Figure 27.7 shows a physical system that can serve as a context for examining this type of problem. The curvature of a slender column subject to an axial load P can be modeled by

d 2y

dx 2 5

M

EI (27.10)

where d2yydx2 specifi es the curvature, M 5 the bending moment, E 5 the modulus of elasticity, and I 5 the moment of inertia of the cross section about its neutral axis. Con- sidering the free body in Fig. 27.7b, it is clear that the bending moment at x is M 5 2Py. Substituting this value into Eq. (27.10) gives

d 2y

dx2 1 p2y 5 0 (27.11)

TF = 1.625

t

TF = 2.815

(a) First mode (b) Second mode

FIGURE 27.6 The principal modes of vibration of two equal masses connected by three identical springs be- tween fi xed walls.

cha9792x_ch27_781-810.indd Page 792 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 792 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 793

where

p2 5 P

EI (27.12)

For the system in Fig. 27.7, subject to the boundary conditions

y(0) 5 0 (27.13a)

y(L) 5 0 (27.13b)

the general solution for Eq. (27.11) is

y 5 A sin(px) 1 B cos(px) (27.14)

where A and B are arbitrary constants that are to be evaluated via the boundary condi- tions. According to the fi rst condition [Eq. (27.13a)],

0 5 A sin(0) 1 B cos(0)

Therefore, we conclude that B 5 0. According to the second condition [Eq. (27.13b)],

0 5 A sin (pL) 1 B cos (pL)

But, since B 5 0, A sin (pL) 5 0. Because A 5 0 represents a trivial solution, we con- clude that sin (pL) 5 0. For this equality to hold,

pL 5 np  for n 5 1, 2, 3, p p (27.15)

Thus, there are an infi nite number of values that meet the boundary condition. Equation (27.15) can be solved for

p 5 np

L   for n 5 1, 2, 3, p (27.16)

which are the eigenvalues for the column.

FIGURE 27.7 (a) A slender rod. (b) A free- body diagram of a rod.

(a)

(0, 0)

P

P�

(L, 0)

x

x

y

y

P�

M

(b)

P

cha9792x_ch27_781-810.indd Page 793 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 793 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

794 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

Figure 27.8, which shows the solution for the fi rst four eigenvalues, can provide insight into the physical signifi cance of the results. Each eigenvalue corresponds to a way in which the column buckles. Combining Eqs. (27.12) and (27.16) gives

P 5 n2p2EI

L2   for n 5 1, 2, 3, p (27.17)

These can be thought of as buckling loads because they represent the levels at which the column moves into each succeeding buckling confi guration. In a practical sense, it is usually the fi rst value that is of interest because failure will usually occur when the column fi rst buckles. Thus, a critical load can be defi ned as

P 5 p2EI

L2

which is formally known as Euler’s formula.

EXAMPLE 27.5 Eigenvalue Analysis of an Axially Loaded Column

Problem Statement. An axially loaded wooden column has the following characteris- tics: E 5 10 3 109 Pa, I 5 1.25 3 1025 m4, and L 5 3 m. Determine the fi rst eight eigenvalues and the corresponding buckling loads.

FIGURE 27.8 The fi rst four eigenvalues for the slender rod from Fig. 27.7.

(a) n = 1

P = � 2EI L2

(b) n = 2

P = 4� 2EI

L2 P = 9�

2EI L2

P = 16� 2EI

L2

(c) n = 3 (d) n = 4

cha9792x_ch27_781-810.indd Page 794 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 794 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 795

Solution. Equations (27.16) and (27.17) can be used to compute

n p, m22 P, kN

1 1.0472 137.078 2 2.0944 548.311 3 3.1416 1233.701 4 4.1888 2193.245 5 5.2360 3426.946 6 6.2832 4934.802 7 7.3304 6716.814 8 8.3776 8772.982

The critical buckling load is, therefore, 137.078 kN.

Although analytical solutions of the sort obtained above are useful, they are often diffi cult or impossible to obtain. This is usually true when dealing with complicated systems or those with heterogeneous properties. In such cases, numerical methods of the sort described next are the only practical alternative.

27.2.4 The Polynomial Method

Equation (27.11) can be solved numerically by substituting a central fi nite-divided-difference approximation (Fig. 23.3) for the second derivative to give

yi11 2 2yi 1 yi21 h2

1 p2yi 5 0

which can be expressed as

yi21 2 (2 2 h2 p2)yi 1 yi11 5 0 (27.18)

Writing this equation for a series of nodes along the axis of the column yields a homo- geneous system of equations. For example, if the column is divided into fi ve segments (that is, four interior nodes), the result is

≥ (2 2 h2p2) 21 0 0

21 (2 2 h2p2) 21 0 0 21 (2 2 h2p2) 21 0 0 21 (2 2 h2p2)

¥ μ y1 y2 y3 y4

∂ 5 0 (27.19)

Expansion of the determinant of the system yields a polynomial, the roots of which are the eigenvalues. This approach, called the polynomial method, is performed in the fol- lowing example.

EXAMPLE 27.6 The Polynomial Method

Problem Statement. Employ the polynomial method to determine the eigenvalues for the axially loaded column from Example 27.5 using (a) one, (b) two, (c) three, and (d) four interior nodes.

cha9792x_ch27_781-810.indd Page 795 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 795 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

796 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

Solution.

(a) Writing Eq. (27.18) for one interior node yields (h 5 3y2)

2(2 2 2.25p2)y1 5 0

Thus, for this simple case, the eigenvalue is analyzed by setting the determinant equal to zero

2 2 2.25p2 5 0

and solving for p 5 60.9428, which is about 10 percent less than the exact value of 1.0472 obtained in Example 27.4.

(b) For two interior nodes (h 5 3y3), Eq. (27.18) is written as

c (2 2 p2) 21 21 (2 2 p2)

d e y1 y2 f 5 0

Expansion of the determinant gives

(2 2 p2)2 2 1 5 0

which can be solved for p 5 61 and 61.73205. Thus, the fi rst eigenvalue is now about 4.5 percent low and a second eigenvalue is obtained that is about 17 percent low.

(c) For three interior points (h 5 3y4), Eq. (27.18) yields

£ 2 2 0.5625p2 21 0

21 2 2 0.5625p2 21

0 21 2 2 0.5625p2 § •

y1 y2 y3

¶ 5 0 (E27.6.1)

The determinant can be set equal to zero and expanded to give

(2 2 0.5625p2)3 2 2(2 2 0.5625p2) 5 0

For this equation to hold, 2 2 0.5625p2 5 0 and 2 2 0.5625p2 5 12. Therefore, the fi rst three eigenvalues can be determined as

p 5 ;1.0205  Zet Z 5 2.5% p 5 ;1.8856  Zet Z 5 10% p 5 ;2.4637  Zet Z 5 22%

(d) For four interior points (h 5 3y5), the result is Eq. (27.19) with 2 2 0.36p2 on the diagonal. Setting the determinant equal to zero and expanding it gives

(2 2 0.36p2)4 2 3(2 2 0.36p2)2 1 1 5 0

which can be solved for the fi rst four eigenvalues

p 5 ;1.0301  Zet Z 5 1.6% p 5 ;1.9593  Zet Z 5 6.5% p 5 ;2.6967  Zet Z 5 14% p 5 ;3.1702  Zet Z 5 24%

cha9792x_ch27_781-810.indd Page 796 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 796 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 797

Table 27.2, which summarizes the results of this example, illustrates some funda- mental aspects of the polynomial method. As the segmentation is made more refi ned, additional eigenvalues are determined and the previously determined values become pro- gressively more accurate. Thus, the approach is best suited for cases where the lower eigenvalues are required.

TABLE 27.2 The results of applying the polynomial method to an axially loaded column. The numbers in parentheses represent the absolute value of the true percent relative error.

Polynomial Method

Eigenvalue True h 5 3/2 h 5 3/3 h 5 3/4 h 5 3/5

1 1.0472 0.9428 1.0000 1.0205 1.0301 (10%) (4.5%) (2.5%) (1.6%) 2 2.0944 1.7321 1.8856 1.9593 (21%) (10%) (65%) 3 3.1416 2.4637 2.6967 (22%) (14%) 4 4.1888 3.1702 (24%)

27.2.5 The Power Method

The power method is an iterative approach that can be employed to determine the largest eigenvalue. With slight modifi cation, it can also be employed to determine the smallest and the intermediate values. It has the additional benefi t that the corresponding eigenvector is obtained as a by-product of the method.

Determination of the Largest Eigenvalue. To implement the power method, the system being analyzed must be expressed in the form

[A]{X} 5 l{X} (27.20)

As illustrated by the following example, Eq. (27.20) forms the basis for an iterative solu- tion technique that eventually yields the highest eigenvalue and its associated eigenvector.

EXAMPLE 27.7 Power Method for Highest Eigenvalue

Problem Statement. Employ the power method to determine the highest eigenvalue for part (c) of Example 27.6.

Solution. The system is fi rst written in the form of Eq. (27.20),

3.556x1 2 1.778x2  5 lx1 21.778x1 1 3.556x2 2 1.778x3 5 lx2

21.778x2 1 3.556x3 5 lx3

cha9792x_ch27_781-810.indd Page 797 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 797 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

798 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

Then, assuming the x’s on the left-hand side of the equation are equal to 1,

3.556(1) 2 1.778(1)  5 1.778

21.778(1) 1 3.556(1) 2 1.778(1) 5 0

21.778(1) 1 3.556(1) 5 1.778

Next, the right-hand side is normalized by 1.778 to make the largest element equal to

• 1.778

0

1.778

¶ 5 1.778 • 1

0

1

¶ Thus, the fi rst estimate of the eigenvalue is 1.778. This iteration can be expressed con- cisely in matrix form as

£ 3.556 21.778 0

21.778 3.556 21.778

0 21.778 3.556

§ • 1

1

1

¶ 5 • 1.778

0

1.778

¶ 5 1.778• 1

0

1

¶ The next iteration consists of multiplying [A] by :1 0 1 ;T to give

£ 3.556 21.778 0

21.778 3.556 21.778

0 21.778 3.556

§ • 1

0

1

¶ 5 • 3.556

23.556

3.556

¶ 5 3.556• 1

21

1

¶ Therefore, the eigenvalue estimate for the second iteration is 3.556, which can be em- ployed to determine the error estimate

Zea Z 5 ` 3.556 2 1.7783.556 ` 100% 5 50% The process can then be repeated. Third iteration:

£ 3.556 21.778 0

21.778 3.556 21.778

0 21.778 3.556

§ • 1

21

1

¶ 5 • 5.334

27.112

5.334

¶ 5 27.112• 20.75

1

20.75

where 0ea 0 5 150% (which is high because of the sign change). Fourth iteration:

£ 3.556 21.778 0

21.778 3.556 21.778

0 21.778 3.556

§ • 20.75

1

20.75

¶ 5 • 24.445

6.223

24.445

¶ 5 6.223• 20.714

1

20.714

where 0ea 0 5 214% (again infl ated because of sign change). Fifth iteration:

£ 3.556 21.778 0

21.778 3.556 21.778

0 21.778 3.556

§ • 20.714

1

20.714

¶ 5 • 24.317

6.095

24.317

¶ 5 6.095• 20.708

1

20.708

cha9792x_ch27_781-810.indd Page 798 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 798 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.2 EIGENVALUE PROBLEMS 799

Thus, the normalizing factor is converging on the value of 6.070 (52.46372) obtained in part (c) of Example 27.6.

Note that there are some instances where the power method will converge to the second- largest eigenvalue instead of to the largest. James, Smith, and Wolford (1985) provide an illustration of such a case. Other special cases are discussed in Fadeev and Fadeeva (1963).

Determination of the Smallest Eigenvalue. There are often cases in engineering where we are interested in determining the smallest eigenvalue. Such was the case for the rod in Fig. 27.7, where the smallest eigenvalue could be used to identify a critical buckling load. This can be done by applying the power method to the matrix inverse of [A]. For this case, the power method will converge on the largest value of 1yl—in other words, the smallest value of l.

EXAMPLE 27.8 Power Method for Lowest Eigenvalue

Problem Statement. Employ the power method to determine the lowest eigenvalue for part (c) of Example 27.6.

Solution. After dividing Eq. E27.6.1 by h2 (5 0.5625), its matrix inverse can be evaluated as

[A]21 5 £ 0.422 0.281 0.141

0.281 0.562 0.281

0.141 0.281 0.422

§ Using the same format as in Example 27.7, the power method can be applied to this matrix. First iteration:

£ 0.422 0.281 0.141

0.281 0.562 0.281

0.141 0.281 0.422

§ • 1

1

1

¶ 5 • 0.884

1.124

0.884

¶ 5 1.124 • 0.751

1

0.751

¶ Second iteration:

£ 0.422 0.281 0.141

0.281 0.562 0.281

0.141 0.281 0.422

§ • 0.751

1

0.751

¶ 5 • 0.704

0.984

0.704

¶ 5 0.984 • 0.715

1

0.715

¶ where 0ea 0 5 14.6%. Third iteration:

£ 0.422 0.281 0.141

0.281 0.562 0.281

0.141 0.281 0.422

§ • 0.715

1

0.715

¶ 5 • 0.684

0.964

0.684

¶ 5 0.964 • 0.709

1

0.709

¶ where 0ea 0 5 4%. Thus, after only three iterations, the result is converging on the value of 0.9602, which is the reciprocal of the smallest eigenvalue, 1.0205(511y0.9602), obtained in Example 27.6c.

cha9792x_ch27_781-810.indd Page 799 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 799 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

800 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

Determination of Intermediate Eigenvalues. After fi nding the largest eigenvalue, it is possible to determine the next highest by replacing the original matrix by one that includes only the remaining eigenvalues. The process of removing the largest known eigenvalue is called defl ation. The technique outlined here, Hotelling’s method, is de- signed for symmetric matrices. This is because it exploits the orthogonality of the eigen- vectors of such matrices, which can be expressed as

{X}Ti {X}j 5 e0 for i ? j1 for i 5 j (27.21) where the components of the eigenvector {X} have been normalized so that {X}T{X} 5 1, that is, so that the sum of the squares of the components equals 1. This can be accom- plished by dividing each of the elements by the normalizing factor

B a n

k51 x2k

Now, a new matrix [A]2 can be computed as

[A]2 5 [A]1 2 l1{X}1{X} T 1 (27.22)

where [A]1 5 the original matrix and l1 5 the largest eigenvalue. If the power method is applied to this matrix, the iteration process will converge to the second largest eigen- value, l2. To show this, fi rst postmultiply Eq. (27.22) by {X}1,

[A]2{X}1 5 [A]1{X}1 2 l1{X}1{X} T 1{X}1

Invoking the orthogonality principle converts this equation to

[A]2{X}1 5 [A]1{X}1 2 l1{X}1

where the right-hand side is equal to zero according to Eq. (27.20). Thus, [A]2{X}1 5 0. Consequently, l 5 0 and {X} 5 {X}1 is a solution to [A]2{X} 5 l{X}. In other words, the [A]2 has eigenvalues of 0, l2, l3, . . . , ln. The largest eigenvalue, l1, has been replaced by a 0 and, therefore, the power method will converge on the next biggest l2. The above process can be repeated by generating a new matrix [A]3, etc. Although in theory this process could be continued to determine the remaining eigenvalues, it is limited by the fact that errors in the eigenvectors are passed along at each step. Thus, it is only of value in determining several of the highest eigenvalues. Although this is some- what of a shortcoming, such information is precisely what is required in many engineer- ing problems.

27.2.6 Other Methods

A wide variety of additional methods are available for solving eigenvalue problems. Most are based on a two-step process. The fi rst step involves transforming the original matrix to a simpler form (for example, tridiagonal) that retains all the original eigenvalues. Then, iterative methods are used to determine these eigenvalues. Many of these approaches are designed for special types of matrices. In particular, a variety of techniques are devoted to symmetric systems. For example, Jacobi’s

cha9792x_ch27_781-810.indd Page 800 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 800 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.3 ODES AND EIGENVALUES WITH SOFTWARE PACKAGES 801

method transforms a symmetric matrix to a diagonal matrix by eliminating off-diagonal terms in a systematic fashion. Unfortunately, the method requires an infi nite number of operations because the removal of each nonzero element often creates a new nonzero value at a previous zero element. Although an infi nite time is required to create all nonzero off-diagonal elements, the matrix will eventually tend toward a diagonal form. Thus, the approach is iterative in that it is repeated until the off-diagonal terms are “suffi ciently” small. Given’s method also involves transforming a symmetric matrix into a simpler form. However, in contrast to the Jacobi method, the simpler form is tridiagonal. In addition, it differs in that the zeros that are created in off-diagonal positions are retained. Conse- quently, it is fi nite and, thus, more effi cient than Jacobi’s method. Householder’s method also transforms a symmetric matrix into a tridiagonal form. It is a fi nite method and is more effi cient than Given’s approach in that it reduces whole rows and columns of off-diagonal elements to zero. Once a tridiagonal system is obtained from Given’s or Householder’s method, the remaining step involves fi nding the eigenvalues. A direct way to do this is to expand the determinant. The result is a sequence of polynomials that can be evaluated iteratively for the eigenvalues. Aside from symmetric matrices, there are also techniques that are available when all eigenvalues of a general matrix are required. These include the LR method of Rutishauser and the QR method of Francis. Although the QR method is less effi cient, it is usually the preferred approach because it is more stable. As such, it is considered to be the best general-purpose solution method. Finally, it should be mentioned that the aforementioned techniques are often used in tandem to capitalize on their respective strengths. For example, Given’s and Householder’s methods can also be applied to nonsymmetric systems. The result will not be tridiagonal but rather a special type called the Hessenberg form. One approach is to exploit the speed of Householder’s approach by employing it to transform the matrix to this form and then use the stable QR algorithm to fi nd the eigenvalues. Additional information on these and other issues related to eigenvalues can be found in Ralston and Rabinowitz (1978), Wilkinson (1965), Fadeev and Fadeeva (1963), and Householder (1953, 1964). Computer codes can be found in a number of sources including Press et al. (2007). Rice (1983) discusses available software packages.

27.3 ODES AND EIGENVALUES WITH SOFTWARE PACKAGES

Software packages have great capabilities for solving ODEs and determining eigenvalues. This section outlines some of the ways in which they can be applied for this purpose.

27.3.1 Excel

Excel’s direct capabilities for solving eigenvalue problems and ODEs are limited. How- ever, if some programming is done (for example, macros), they can be combined with Excel’s visualization and optimization tools to implement some interesting applications. Section 28.1 provides an example of how the Excel Solver can be used for parameter estimation of an ODE.

S O F T W A R E

cha9792x_ch27_781-810.indd Page 801 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 801 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

802 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

27.3.2 MATLAB

As might be expected, the standard MATLAB software package has excellent capa- bilities for determining eigenvalues and eigenvectors. However, it also has built-in functions for solving ODEs. The standard ODE solvers include two functions to im- plement the adaptive step-size Runge-Kutta Fehlberg method (recall Sec. 25.5.2). These are ode23, which uses second- and third-order formulas to attain medium accuracy, and ode45, which uses fourth- and fi fth-order formulas to attain higher accuracy. The following example illustrates how they can be used to solve a system of ODEs.

EXAMPLE 27.9 Using MATLAB for Eigenvalues and ODEs

Problem Statement. Explore how MATLAB can be used to solve the following set of nonlinear ODEs from t 5 0 to 20:

dx

dt 5 1.2x 2 0.6x y  

dy

dt 5 20.8y 1 0.3xy

where x 5 2 and y 5 1 at t 5 0. As we will see in the next chapter (Sec. 28.2), such equations are referred to as predator-prey equations.

Solution. Before obtaining a solution with MATLAB, you must use a text processor to create an M-fi le containing the right-hand side of the ODEs. This M-fi le will then be accessed by the ODE solver [where x 5 y(1) and y 5 y(2)]:

function yp = predprey(t,y) yp = [1.2*y(1) –0.6*y(1)*y(2); –0.8*y(2)+0.3*y(1)*y(2)];

We stored this M-fi le under the name: predprey.m. Next, start up MATLAB, and enter the following commands to specify the integra- tion range and the initial conditions:

>> tspan = [0,20]; >> y0 = [2,1];

The solver can then be invoked by

>> [t,y]=ode23('predprey',tspan,y0);

This command will then solve the differential equations in predprey.m over the range defi ned by tspan using the initial conditions found in y0. The results can be displayed by simply typing

>> plot(t,y)

which yields Fig. 27.9.

S O F T W A R E

cha9792x_ch27_781-810.indd Page 802 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 802 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.3 ODES AND EIGENVALUES WITH SOFTWARE PACKAGES 803

In addition, it is also instructive to generate a state-space plot, that is, a plot of the dependent variables versus each other by

>> plot(y(:,1),y(:,2))

which yields Fig. 27.10.

FIGURE 27.9 Solution of predator-prey model with MATLAB.

FIGURE 27.10 State-space plot of predator-prey model with MATLAB.

MATLAB also has a range of functions designed for stiff systems. These include ode15s and ode23s. As in the following example, they succeed where the standard functions fail.

cha9792x_ch27_781-810.indd Page 803 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 803 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

804 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

EXAMPLE 27.10 MATLAB for Stiff ODEs

Problem Statement. Van der Pol’s equation can be written as

dy1 dt

5 y2

dy2 dt

5 m(1 2 y21)y2 2 y1

As the parameter m gets large, the system becomes progressively stiffer. Given the initial conditions, y1(0) 5 y2(0) 5 1, use MATLAB to solve the following two cases

(a) For m 5 1, use ode45 to solve from t 5 0 to 20. (b) For m 5 1000, use ode23s to solve from t 5 0 to 3000.

Solution.

(a) An M-fi le can be created to hold the differential equations,

function yp = vanderpol(t,y) yp=[y(2);1*(1–y(1)^2)*y(2)–y(1)];

Then, as in Example 27.9, ode45 can be invoked and the results plotted (Fig. 27.11),

>> tspan=[0,20]; >> y0=[1,1]; >> [t,y]=ode45('vanderpol',tspan,y0); >> plot(t,y(:,1))

(b) If a standard solver like ode45 is used for the stiff case (m 5 1000), it will fail miser- ably (try it, if you like). However, ode23s does an effi cient job. After revising the M-fi le to refl ect the new value of m, the solution can be obtained and graphed (Fig. 27.12),

>> tspan=[0,3000]; >> y0=[1,1];

FIGURE 27.11 Nonstiff form of Van der Pol’s equation solved with MATLAB’s ode45 function.

S O F T W A R E

cha9792x_ch27_781-810.indd Page 804 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 804 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.3 ODES AND EIGENVALUES WITH SOFTWARE PACKAGES 805

>> [t,y]=ode23S('vanderpol',tspan,y0); >> plot(t,y(:,1))

Notice how this solution has much sharper edges than for case (a). This is a visual manifestation of the “stiffness” of the solution.

FIGURE 27.12 Stiff form of Van der Pol’s equation solved with MATLAB’s ode23s function.

For eigenvalues, the capabilities are also very easy to apply. Recall that, in our discus- sion of stiff systems in Chap. 26, we presented the stiff system defi ned by Eq. (26.6). Such linear ODEs can be written as an eigenvalue problem of the form

c5 2 l 23 2100 301 2 l

d e e1 e2 f 5 {0}

where l and {e} 5 the eigenvalue and eigenvector, respectively. MATLAB can then be employed to evaluate both the eigenvalues (d) and eigenvec- tors (v) with the following simple commands:

>> a=[5 –3;–100 301]; >> [v,d]=eig(a)

v = –0.9477 0.0101 –0.3191 –0.9999

d = 3.9899 0 0 302.0101

Thus, we see that the eigenvalues are of quite different magnitudes, which is typical of a stiff system.

cha9792x_ch27_781-810.indd Page 805 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 805 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

806 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

The eigenvalues can be interpreted by recognizing that the general solution for a system of ODEs can be represented as the sum of exponentials. For example, the solution for the present case would be of the form

y1 5 c11e 23.9899t 1 c12e

2302.0101t

y2 5 c21e 23.9899t 1 c22e

2302.0101t

where cij 5 the part of the initial condition for yi that is associated with the jth eigenvalue. It should be noted that the c’s can be evaluated from the initial conditions and the eigenvectors. Any good book on differential equations, for example, Boyce and DiPrima (1992), will provide an explanation of how this can be done. Because, for the present case, all the eigenvalues are positive (and hence negative in the exponential function), the solution consists of a series of decaying exponentials. The one with the largest eigenvalue (in this case, 302.0101) would dictate the step size if an explicit solution technique were used.

27.3.3 Mathcad

Mathcad has a number of different functions that solve differential equations and deter- mine eigenvalues and eigenvectors. The most basic technique employed by Mathcad to solve systems of fi rst-order differential equations is a fi xed step-size fourth-order Runge- Kutta algorithm. This is provided by the rkfi xed function. Although this is a good all- purpose integrator, it is not always effi cient. Therefore, Mathcad supplies Rkadapt, which is a variable step sized version of rkfi xed. It is well suited for functions that change rapidly in some regions and slowly in others. Similarly, if you know your solu- tion is a smooth function, then you may fi nd that the Mathcad Bulstoer function works well. This function employs the Bulirsch-Stoer method and is often both effi cient and highly accurate for smooth functions. Stiff differential equations are at the opposite end of the spectrum. Under these conditions the rkfi xed function may be very ineffi cient or unstable. Therefore, Mathcad provides two special methods specifi cally designed to handle stiff systems. These func- tions are called Stiffb and Stiffr and are based on a modifi ed Bulirsch-Stoer method for stiff systems and the Rosenbrock method. As an example, let’s use Mathcad to solve the following nonlinear ODEs,

dy1 dt

5 1.2y1 2 0.6y1 y2

dy2 dt

5 20.8y2 1 0.3y1 y2

with the initial conditions, y1 5 2 and y2 5 1. This system, called Lotka-Volterra equa- tions, are used by environmental engineers and ecologists to evaluate the interactions of predators (y2) and prey (y1). As in Fig. 27.13, the defi nition symbol is fi rst used to defi ne the vector D(u, y) holding the right-hand sides of the ODEs for input to rkfi xed. Note that y1 and y2 in the ODEs are changed to y0 and y1 to comply with Mathcad requirements. In addition,

S O F T W A R E

cha9792x_ch27_781-810.indd Page 806 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 806 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.3 ODES AND EIGENVALUES WITH SOFTWARE PACKAGES 807

we defi ne the initial conditions (y0), the integration limit (tf) and the number of values we want to generate (npts). The solutions for rkfi xed with 200 steps between t 5 0 and tf are stored in the ysol matrix. The solution is displayed graphically in the plot in Fig. 27.13. Next, we can illustrate how Mathcad evaluates eigenvalues and eigenvectors. The function eigenvals(M) returns the eigenvalues of the square matrix M. The function eigenvecs(M) returns a matrix containing normalized eigenvectors corresponding to the eigenvectors of M whereas eigenvec(M,e) returns the eigenvector corresponding to the eigenvalue e. We can illustrate these functions for the system given by [recall Eq. (26.6)]

dy1 dt

5 25y1 1 3y2

dy2 dt

5 100y1 2 301y2

The results are shown in Fig. 27.14. Because the eigenvalues (aa) are of different magnitudes, the system is stiff. Note that bb holds the specifi c eigenvector associated with the smaller eigenvalue. The result cc is a matrix containing both eigenvectors as its columns.

FIGURE 27.13 Mathcad screen to solve a system of ODEs.

cha9792x_ch27_781-810.indd Page 807 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 807 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

808 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

FIGURE 27.14 Mathcad screen to solve for the eigenvalues of a system of ODEs.

S O F T W A R E

PROBLEMS

27.1 A steady-state heat balance for a rod can be represented as

d 2T

dx2 2 0.15T 5 0

Obtain an analytical solution for a 10-m rod with T(0) 5 240 and T(10) 5 150. 27.2 Use the shooting method to solve Prob. 27.1. 27.3 Use the fi nite-difference approach with Dx 5 1 to solve Prob. 27.1. 27.4 Use the shooting method to solve

7 d 2y

dx2 2 2

dy

dx 2 y 1 x 5 0

with the boundary conditions y(0) 5 5 and y(20) 5 8. 27.5 Solve Prob. 27.4 with the fi nite-difference approach using Dx 5 2. 27.6 Use the shooting method to solve

d 2T

dx2 2 1 3 1027(T 1 273)4 1 4(150 2 T) 5 0 (P27.6.1)

Obtain a solution for boundary conditions: T(0) 5 200 and T(0.5) 5 100. 27.7 Differential equations like the one solved in Prob. 27.6 can often be simplifi ed by linearizing their nonlinear terms. For example, a fi rst-order Taylor series expansion can be used to linearize the quartic term in Eq. (P27.6.1) as

1 3 1027(T 1 273)4 5 1 3 1027(Tb 1 273) 4 1 4

3 1027(Tb 1 273) 3(T 2 Tb)

where Tb is a base temperature about which the term is linearized. Substitute this relationship into Eq. (P27.6.1), and then solve the resulting linear equation with the fi nite-difference approach. Employ Tb 5 150 and Dx 5 0.01 to obtain your solution. 27.8 Repeat Example 27.4 but for three masses. Produce a plot like Fig. 27.6 to identify the principle modes of vibration. Change all the k’s to 240. 27.9 Repeat Example 27.6, but for fi ve interior points (h 5 3y6).

cha9792x_ch27_781-810.indd Page 808 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 808 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 809

dz

dt 5 2bz 1 xy

where s 5 10, b 5 2.666667, and r 5 28. Employ initial condi- tions of x 5 y 5 z 5 5 and integrate from t 5 0 to 20. 27.23 Use fi nite differences to solve the boundary-value ordinary differential equation

d 2u

dx2 1 6

du

dx 2 u 5 2

with boundary conditions u(0) 5 10 and u(2) 5 1. Plot the results of u versus x. Use Dx 5 0.1. 27.24 Solve the nondimensionalized ODE using fi nite difference methods that describe the temperature distribution in a circular rod with internal heat source S

d 2T

dr2 1

1 r dT

dr 1 S 5 0

over the range 0 # r # 1, with the boundary conditions

T(r 5 1) 5 1   dT

dr ` r50

5 0

for S 5 1, 10, and 20 K/m2. Plot the temperature versus radius. 27.25 Derive the set of differential equations for a three mass–four spring system (Fig. P27.25) that describes their time motion. Write the three differential equations in matrix form,

[Acceleration vector]1[kym matrix][displacement vector x] 50

Note each equation has been divided by the mass. Solve for the eigenvalues and natural frequencies for the following values of mass and spring constants: k1 5 k4 5 15 N/m, k2 5 k3 5 35 N/m, and m1 5 m2 5 m3 5 1.5 kg.

27.10 Use minors to expand the determinant of

£ 2 2 l 8 10

8 4 2 l 5

10 5 7 2 l

§

27.11 Use the power method to determine the highest eigenvalue and corresponding eigenvector for Prob. 27.10. 27.12 Use the power method to determine the lowest eigenvalue and corresponding eigenvector for Prob. 27.10. 27.13 Develop a user-friendly computer program to implement the shooting method for a linear second-order ODE. Test the program by duplicating Example 27.1. 27.14 Use the program developed in Prob. 27.13 to solve Probs. 27.2 and 27.4. 27.15 Develop a user-friendly computer program to implement the fi nite-difference approach for solving a linear second-order ODE. Test it by duplicating Example 27.3. 27.16 Use the program developed in Prob. 27.15 to solve Probs. 27.3 and 27.5. 27.17 Develop a user-friendly program to solve for the largest eigen- value with the power method. Test it by duplicating Example 27.7. 27.18 Develop a user-friendly program to solve for the smallest ei- genvalue with the power method. Test it by duplicating Example 27.8. 27.19 Use the Excel Solver to directly solve (that is, without lin- earization) Prob. 27.6 using the fi nite-difference approach. Employ Dx 5 0.1 to obtain your solution. 27.20 Use MATLAB to integrate the following pair of ODEs from t 5 0 to 100:

dy1 dt

5 0.35y1 2 1.6y1y2   dy2 dt

5 0.04y1y2 2 0.15y2

where y1 5 1 and y2 5 0.05 at t 5 0. Develop a state-space plot (y1 versus y2) of your results. 27.21 The following differential equation can be used to analyze the vibrations of an automobile shock absorber:

1.25 3 106 d 2x

dt2 1 1 3 107

dx

dt 1 1.5 3 109x 5 0

Transform this equation into a pair of ODEs. (a) Use MATLAB to solve these equations from t 5 0 to 0.4 for the case where x 5 0.5, and dxydt 5 0 at t 5 0. (b) Use MATLAB to determine the eigen- values and eigenvectors for the system. 27.22 Use MATLAB or Mathcad to integrate

dx

dt 5 2sx 1 sy

dy

dt 5 rx 2 y 2 xz

FIGURE P27.25

k2 k3 k4k1

x1 x2 x3

m1 m2 m3

27.26 Consider the mass-spring system in Fig. P27.26. The fre- quencies for the mass vibrations can be determined by solving for the eigenvalues and by applying M  x

$ 1 kx 5 0, which yields

£ m1 0 0

0 m2 0

0 0 m3

§ • x $

1

x $

2

x $

3

¶ 1 £ 2k 2k 2k

2k 2k 2k

2k 2k 2k

§ • x1 x2 x3

¶ 5 • 0

0

0

cha9792x_ch27_781-810.indd Page 809 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 809 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

810 BOUNDARY-VALUE AND EIGENVALUE PROBLEMS

(b) Using the fourth-order RK method with a constant step size of 0.03125.

(c) Using the MATLAB function ode45. (d) Using the MATLAB function ode23s. (e) Using the MATLAB function ode23tb. Present your results in graphical form. 27.28 A heated rod with a uniform heat source can be modeled with the Poisson equation,

d 2T

dx2 5 2f (x)

Given a heat source f(x) 5 25 and the boundary conditions, T(x 5 0) 5 40 and T(x 5 10) 5 200, solve for the temperature distri- bution with (a) the shooting method and (b) the fi nite-difference method ( Dx 5 2). 27.29 Repeat Prob. 27.28, but for the following heat source: f(x) 5 0.12x3 2 2.4x2 1 12x. 27.30 Suppose that the position of a falling object is governed by the following differential equation,

d 2x

dt2 1

c

m dx

dt 2 g 5 0

where c 5 a fi rst-order drag coeffi cient 5 12.5 kg/s, m 5 mass 5 70 kg, and g 5 gravitational acceleration 5 9.81 m/s2. Use the shooting method to solve this equation for position and velocity given the boundary conditions, x(0) 5 0 and x(12) 5 500. 27.31 Repeat Example 27.3, but insulate the left end of the rod. That is, change the boundary condition at the left end of the rod to T9(0) 5 0.

Applying the guess x 5 x0e ivt as a solution, we get the following

matrix:

£ 2k 2 m1v

2 2k 2k

2k 2k2m2v 2 2k

2k 2k 2k2m3v 2

§ • x01 x02 x03

¶ eivt 5 • 0

0

0

Use MATLAB’s eig command to solve for the eigenvalues of the k 2 mv2 matrix above. Then use these eigenvalues to solve for the frequencies (v). Let m1 5 m2 5 m3 5 1 kg, and k 5 2 N/m.

FIGURE P27.26

k

k

k

x1 x2 x3

m1 m2 m3

27.27 The following nonlinear, parasitic ODE was suggested by Hornbeck (1975):

dy1 dt

5 5(y1 2 t 2)

If the initial condition is y1(0) 5 0.08, obtain a solution from t 5 0 to 5: (a) Analytically.

cha9792x_ch27_781-810.indd Page 810 29/10/13 9:51 PM F-468 cha9792x_ch27_781-810.indd Page 810 29/10/13 9:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28 C H A P T E R 28

811

Case Studies: Ordinary Differential Equations

The purpose of this chapter is to solve some ordinary differential equations using the numerical methods presented in Part Seven. The equations originate from practical en- gineering applications. Many of these applications result in nonlinear differential equa- tions that cannot be solved using analytic techniques. Therefore, numerical methods are usually required. Thus, the techniques for the numerical solution of ordinary differential equations are fundamental capabilities that characterize good engineering practice. The problems in this chapter illustrate some of the trade-offs associated with various methods developed in Part Seven. Section 28.1 derives from a chemical engineering problem context. It demonstrates how the transient behavior of chemical reactors can be simulated. It also illustrates how optimization can be used to estimate parameters for ODEs. Sections 28.2 and 28.3, which are taken from civil and electrical engineering, re- spectively, deal with the solution of systems of equations. In both cases, high accuracy is demanded, and as a consequence, a fourth-order RK scheme is used. In addition, the electrical engineering application also deals with determining eigenvalues. Section 28.4 employs a variety of different approaches to investigate the behavior of a swinging pendulum. This problem also utilizes two simultaneous equations. An important aspect of this example is that it illustrates how numerical methods allow nonlinear effects to be incorporated easily into an engineering analysis.

28.1 USING ODES TO ANALYZE THE TRANSIENT RESPONSE OF A REACTOR (CHEMICAL/BIO ENGINEERING)

Background. In Sec. 12.1, we analyzed the steady state of a series of reactors. In ad- dition to steady-state computations, we might also be interested in the transient response of a completely mixed reactor. To do this, we have to develop a mathematical expression for the accumulation term in Eq. (12.1). Accumulation represents the change in mass in the reactor per change in time. For a constant-volume system, it can be simply formulated as

Accumulation 5 V dc

dt (28.1)

cha9792x_ch28_811-840.indd Page 811 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 811 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

812 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

where V 5 volume and c 5 concentration. Thus, a mathematical formulation for accu- mulation is volume times the derivative of c with respect to t. In this application we will incorporate the accumulation term into the general mass- balance framework we developed in Sec. 12.1. We will then use it to simulate the dynamics of a single reactor and a system of reactors. In the latter case, we will show how the system’s eigenvalues can be determined and provide insight into its dynamics. Finally, we will illustrate how optimization can be used to estimate the parameters of mass-balance models.

Solution. Equations (28.1) and (12.1) can be used to represent the mass balance for a single reactor such as the one shown in Fig. 28.1:

V dc

dt 5 Qcin 2 Qc (28.2)

Accumulation 5 inputs 2 outputs

Equation (28.2) can be used to determine transient or time-variable solutions for the reactor. For example, if c 5 c0 at t 5 0, calculus can be employed to analytically solve Eq. (28.2) for

c 5 cin(1 2 e2(QyV)t) 1 c0e2(QyV)t

If cin 5 50 mg/m 3, Q 5 5 m3/min, V 5 100 m3, and c0 5 10 mg/m

3, the equation is

c 5 50(1 2 e20.05t) 1 10e20.05t

Figure 28.2 shows this exact, analytical solution. Euler’s method provides an alternative approach for solving Eq. (28.2). Figure 28.2 includes two solutions with different step sizes. As the step size is decreased, the nu- merical solution converges on the analytical solution. Thus, for this case, the numerical method can be used to check the analytical result. Besides checking the results of an analytical solution, numerical solutions have added value in those situations where analytical solutions are impossible or so diffi cult that they are impractical. For example, aside from a single reactor, numerical methods have utility when simulating the dynamics of systems of reactors. For example, ODEs

Qc

Qcin

FIGURE 28.1 A single, completely mixed reactor with an infl ow and an outfl ow.

cha9792x_ch28_811-840.indd Page 812 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 812 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.1 USING ODES TO ANALYZE THE TRANSIENT RESPONSE OF A REACTOR 813

can be written for the fi ve coupled reactors in Fig. 12.3. The mass balance for the fi rst reactor can be written as

V1 dc1 dt

5 Q01c01 1 Q31c3 2 Q12c1 2 Q15c1

or, substituting parameters (note that Q01c01 5 50 mg/min, Q03c03 5 160 mg/min, V1 5 50 m3, V2 5 20 m

3, V3 5 40 m 3, V4 5 80 m

3, and V5 5 100 m 3),

dc1 dt

5 20.12c1 1 0.02c3 1 1

Similarly, balances can be developed for the other reactors as

dc2 dt

5 0.15c1 2 0.15c2

dc3 dt

5 0.025c2 2 0.225c3 1 4

dc4 dt

5 0.1c3 2 0.1375c4 1 0.025c5

dc5 dt

5 0.03c1 1 0.01c2 2 0.04c5

Suppose that at t 5 0 all the concentrations in the reactors are at zero. Compute how their concentrations will increase over the next hour. The equations can be integrated with the fourth-order RK method for systems of equations and the results are depicted in Fig. 28.3. Notice that each of the reactors shows a different transient response to the introduction of chemical. These responses can be parameterized by a 90 percent response time t90, which measures the time required for each reactor to reach 90 percent of its ultimate steady-state level. The times range from

FIGURE 28.2 Plot of analytical and numerical solutions of Eq. (28.2). The numerical solutions are obtained with Euler’s method using different step sizes.

c, m

g /m

3

0 10 20 t, min

30 50

10

0

30

50

20

40

40

Euler, step size = 10 step size = 5

Exact

cha9792x_ch28_811-840.indd Page 813 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 813 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

814 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

t90

c–1

0 t

c1

10

t90

c–3

0 t

c3

10

t90

c–4

0 t

c4

10

t90

c–2

0 t

c2

10

t90

c–5

0 500 t

c5

10

FIGURE 28.3 Plots of transient or dynamic response of the network of reactors from Fig. 12.3. Note that all the reactors eventually approach their steady-state concentrations previously computed in Sec. 12.1. In addition, the time to steady state is parameterized by the 90 percent response time t90.

cha9792x_ch28_811-840.indd Page 814 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 814 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.1 USING ODES TO ANALYZE THE TRANSIENT RESPONSE OF A REACTOR 815

about 10 min for reactor 3 to about 70 min for reactor 5. The response times of reactors 4 and 5 are of particular concern because the two outfl ow streams for the system exit these tanks. Thus, a chemical engineer designing the system might change the fl ows or volumes of the reactors to speed up the response of these tanks while still maintaining the desired outputs. Numerical methods of the sort described in this part of the book can prove useful in these design calculations. Further insight into the system’s response characteristics can be developed by computing its eigenvalues. First, the system of ODEs can be written as an eigenvalue problem as

E0.12 2 l 0 20.02 0 020.15 0.15 2 l 0 0 00 20.025 0.225 2 l 0 0 0 0 20.1 0.1375 2 l 20.025

20.03 20.01 0 0 0.04 2 l

U ee1e2e3 e4 e5

u 5 {0} where l and {e} 5 the eigenvalue and the eigenvector, respectively. A package like MATLAB software can be used to very conveniently generate the eigenvalues and eigenvectors,

>> a=[0.12 0.0 –0.02 0.0 0.0;–.15 0.15 0.0 0.0 0.0;0.0 –0.025 0.225 0.0 0.0; 0.0 0.0 –.1 0.1375 –0.025;–0.03 –0.01 0.0 0.0 0.04];

>> [e,l]=eig(a) e = 0 0 –0.1228 –0.1059 0.2490 0 0 0.2983 0.5784 0.8444 0 0 0.5637 0.3041 0.1771 1.0000 0.2484 –0.7604 –0.7493 0.3675 0 0.9687 0.0041 –0.0190 –0.2419

l = 0.1375 0 0 0 0 0 0.0400 0 0 0 0 0 0.2118 0 0 0 0 0 0.1775 0 0 0 0 0 0.1058

The eigenvalues can be interpreted by recognizing that the general solution for a system of ODEs can be represented as the sum of exponentials. For example, for reactor 1, the general solution would be of the form

c1 5 c11e 2l1t 1 c12e

2l2t 1 c13e 2l3t 1 c14e

2l4t 1 c15e 2l5t

where cij 5 the part of the initial condition for reactor i that is associated with the jth eigenvalue. Thus, because, for the present case, all the eigenvalues are positive (and hence negative in the exponential function), the solution consists of a series of decaying exponentials. The one with the smallest eigenvalue (in our case, 0.04) will be the slowest.

cha9792x_ch28_811-840.indd Page 815 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 815 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

816 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

FIGURE 28.4 A simple experiment to collect rate data for a chemical compound that decays with time (reprinted from Chapra 1997).

c

t

t = 0 t = 2 t = 3t = 1

c0 c2 c3c1

Time Concentration

0 c0

1 c1

2 c2

3 c3

In some cases, the engineer performing this analysis could be able to relate this eigen- value back to the system parameters. For example, the ratio of the outfl ow from reactor 5 to its volume is (Q55 1 Q54)yV5 5 4y100 5 0.04. Such information can then be used to modify the system’s dynamic performance. The fi nal topic we would like to review within the present context is parameter estimation. One area where this occurs frequently is in reaction kinetics, that is, the quantifi cation of chemical reaction rates. A simple example is depicted in Fig. 28.4. A series of beakers are set up containing a chemical compound that decays over time. At time intervals, the concentration in one of the beakers is measured and recorded. Thus, the result is a table of times and concentrations. One model that is commonly used to describe such data is

dc

dt 5 2kcn (28.3)

where k 5 a reaction rate and n 5 the order of the reaction. Chemical engineers use concentration-time data of the sort depicted in Fig. 28.4 to estimate k and n. One way to do this is to guess values of the parameters and then solve Eq. (28.3) numerically. The predicted values of concentration can be compared with the measured concentrations and an assessment of the fi t made. If the fi t is deemed inadequate (for example, by ex- amining a plot or a statistical measure like the sum of the squares of the residuals), the guesses are adjusted and the procedure repeated until a decent fi t is attained. The following data can be fi t in this fashion:

t, d 0 1 3 5 10 15 20

c, mg/L 12 10.7 9 7.1 4.6 2.5 1.8

cha9792x_ch28_811-840.indd Page 816 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 816 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.1 USING ODES TO ANALYZE THE TRANSIENT RESPONSE OF A REACTOR 817

A B C D E F G H 1 Fitting of reaction rate 2 data with the integral/least-squares approach 3 k 0.091528 4 n 1.044425 5 dt 1 6 t k1 k2 k3 k4 cp cm (cp-cm)^2 7 0 21.22653 21.16114 21.16462 21.10248 12 12 0 8 1 21.10261 21.04409 21.04719 20.99157 10.83658 10.7 0.018653 9 2 20.99169 20.93929 20.94206 20.89225 9.790448 10 3 20.89235 20.84541 20.84788 20.80325 8.849344 9 0.022697 11 4 20.80334 20.76127 20.76347 20.72346 8.002317 12 5 20.72354 20.68582 20.68779 20.65191 7.239604 7.1 0.019489 13 6 20.65198 20.61814 20.61989 20.5877 6.552494 14 7 20.58776 20.55739 20.55895 20.53005 5.933207 15 8 20.53011 20.50283 20.50424 20.47828 5.374791 16 9 20.47833 20.45383 20.45508 20.43175 4.871037 17 10 20.4318 20.40978 20.4109 20.38993 4.416389 4.6 0.033713 18 11 20.38997 20.37016 20.37117 20.35231 4.005877 19 12 20.35234 20.33453 20.33543 20.31846 3.635053 20 13 20.31849 20.30246 20.30326 20.28798 3.299934 21 14 20.28801 20.27357 20.2743 20.26054 2.996949 22 15 20.26056 20.24756 20.24821 20.23581 2.7229 2.5 0.049684 23 16 20.23583 20.22411 20.22469 20.21352 2.474917 24 17 20.21354 20.20297 20.20349 20.19341 2.250426 25 18 20.19343 20.18389 20.18436 20.17527 2.047117 26 19 20.17529 20.16668 20.16711 20.1589 1.862914 27 20 20.15891 20.15115 20.15153 20.14412 1.695953 1.8 0.010826 28 29 SSR 5 0.155062

FIGURE 28.5 The application of a spreadsheet and numerical methods to determine the order and rate coeffi cient of reaction data. This application was performed with the Excel spreadsheet.

The solution to this problem is shown in Fig. 28.5. The Excel spreadsheet was used to perform the computation. Initial guesses for the reaction rate and order are entered into cells B3 and B4, re- spectively, and the time step for the numerical calculation is typed into cell B5. For this case, a column of calculation times is entered into column A starting at 0 (cell A7) and ending at 20 (cell A27). The k1 through k4 coeffi cients of the fourth-order RK method are then calculated in the block B7..E27. These are then used to determine the predicted concentrations (the cp values) in column F. The measured values (cm) are entered in column G adjacent to the corresponding predicted values. These are then used in con- junction with the predicted values to compute the squared residual in column H. These values are then summed in cell H29. At this point, the Excel Solver can be used to determine the best parameter values. Once you have accessed the Solver, you are prompted for a target or solution cell (H29), queried whether you want to maximize or minimize the target cell (minimize), and

cha9792x_ch28_811-840.indd Page 817 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 817 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

818 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

prompted for the cells that are to be varied (B3..B4). You then activate the algorithm [s(olve)], and the results are as in Fig. 28.5. As shown, the values in cells B3..B4 (k 5 0.0915 and n 5 1.044) minimize the sum of the squares of the residuals (SSR 5 0.155) between the predicted and measured data. A plot of the fi t along with the data is shown in Fig. 28.6.

28.2 PREDATOR-PREY MODELS AND CHAOS (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. Environmental engineers deal with a variety of problems involving sys- tems of nonlinear ordinary differential equations. In this section, we will focus on two of these applications. The fi rst relates to the so-called predator-prey models that are used to study the cycling of nutrient and toxic pollutants in aquatic food chains and biological treatment systems. The second are equations derived from fl uid dynamics that are used to simulate the atmosphere. Aside from their obvious application to weather prediction, such equations have also been used to study air pollution and global climate change. Predator-prey models were developed independently in the early part of the twentieth century by the Italian mathematician Vito Volterra and the American biologist Alfred J. Lotka. These equations are commonly called Lotka-Volterra equations. The simplest example is the following pair of ODEs:

dx

dt 5 ax 2 bxy (28.4)

dy

dt 5 2cy 1 dxy (28.5)

where x and y 5 the number of prey and predators, respectively, a 5 the prey growth rate, c 5 the predator death rate, and b and d 5 the rate characterizing the effect of the predator-prey interaction on prey death and predator growth, respectively. The multiplica- tive terms (that is, those involving xy) are what make such equations nonlinear.

FIGURE 28.6 Plot of fi t generated with the integral/least-squares approach.

10

c

5

0 t20100

cha9792x_ch28_811-840.indd Page 818 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 818 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.2 PREDATOR-PREY MODELS AND CHAOS 819

An example of a simple model based on atmospheric fl uid dynamics is the Lorenz equations developed by the American meteorologist Edward Lorenz,

dx

dt 5 2sx 1 sy (28.6)

dy

dt 5 rx 2 y 2 xz (28.7)

dz

dt 5 2bz 1 xy (28.8)

Lorenz developed these equations to relate the intensity of atmospheric fl uid motion, x, to temperature variations y and z in the horizontal and vertical directions, respectively. As with the predator-prey model, we see that the nonlinearity is localized in simple multiplicative terms (xz and xy). Use numerical methods to obtain solutions for these equations. Plot the results to visualize how the dependent variables change temporally. In addition, plot the dependent variables versus each other to see whether any interesting patterns emerge.

Solution. Use the following parameter values for the predator-prey simulation: a 5 1.2, b 5 0.6, c 5 0.8, and d 5 0.3. Employ initial conditions of x 5 2 and y 5 1 and inte- grate from t 5 0 to 30. We will use the fourth-order RK method with double precision to obtain solutions. The results using a step size of 0.1 are shown in Fig. 28.7. Note that a cyclical pat- tern emerges. Thus, because predator population is initially small, the prey grows expo- nentially. At a certain point, the prey become so numerous, that the predator population begins to grow. Eventually, the increased predators cause the prey to decline. This de- crease, in turn, leads to a decrease of the predators. Eventually, the process repeats. Notice that, as expected, the predator peak lags the prey. Also, observe that the process has a fi xed period, that is, it repeats in a set time. Now, if the parameters used to simulate Fig. 28.7 were changed, although the gen- eral pattern would remain the same, the magnitudes of the peaks, lags, and period would change. Thus, there are an infi nite number of cycles that could occur. A phase-plane representation is useful in discerning the underlying structure of the model. Rather than plotting x and y versus t, we can plot x versus y. This plot illustrates

FIGURE 28.7 Time-domain representation of numbers of prey and predators for the Lotka-Volterra model.

8

4

0 t30200 10

x, prey y, predator

cha9792x_ch28_811-840.indd Page 819 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 819 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

820 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

the way that the state variables (x and y) interact, and is referred to as a phase-plane representation. Figure 28.8 shows the phase-plane representation for the case we are studying. Thus, the interaction between the predator and the prey defi nes a closed counterclockwise orbit. Notice that there is a critical or rest point at the center of the orbit. The exact location of this point can be determined by setting Eqs. (28.4) and (28.5) to steady state (dyydt 5 dxydt 5 0) and solving for (x, y) 5 (0, 0) and (cyd, ayb). The former is the trivial result that if we start with neither predators nor prey, nothing will happen. The latter is the more interesting outcome that if the initial conditions are set at x 5 cyd and y 5 ayb, the derivative will be zero and the populations will remain constant. Now, let us use the same approach to investigate the trajectories of the Lorenz equa- tions with the following parameter values: s 5 10, b 5 2.666667, and r 5 28. Employ initial conditions of x 5 y 5 z 5 5 and integrate from t 5 0 to 20. Again, we will use the fourth-order RK method with double precision to obtain solutions. The results shown in Fig. 28.9 are quite different from the behavior of the Lotka- Volterra equations. The variable x seems to be undergoing an almost random pattern of oscillations, bouncing around from negative values to positive values. However, even

FIGURE 28.8 Phase-plane representation for the Lotka-Volterra model. 4

y

2

0 x62

Critical point

0 4

FIGURE 28.9 Time-domain representation of x versus t for the Lorenz equations. The solid time series is for the initial conditions (5, 5, 5). The dotted line is where the initial condition for x is perturbed slightly (5.001, 5, 5).

20

x

0

10

– 20

– 10

20 t155 10

cha9792x_ch28_811-840.indd Page 820 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 820 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.2 PREDATOR-PREY MODELS AND CHAOS 821

though the patterns seem random, the frequency of the oscillation and the amplitudes seem fairly consistent. Another interesting feature can be illustrated by changing the initial condition for x slightly (from 5 to 5.001). The results are superimposed as the dotted line in Fig. 28.9. Although the solutions track on each other for a time, after about t 5 12.5 they diverge signifi cantly. Thus, we can see that the Lorenz equations are quite sensitive to their initial conditions. In his original study, this led Lorenz to the conclusion that long-range weather forecasts might be impossible! Finally, let us examine the phase-plane plots. Because we are dealing with three in- dependent variables, we are limited to projections. Figure 28.10 shows projections in the xy and the xz planes. Notice how a structure is manifest when perceived from the phase- plane perspective. The solution forms orbits around what appear to be critical points.

FIGURE 28.10 Phase-plane representation for the Lorenz equations. (a) xy projection and (b) xz projection.

25 y

– 25

x20– 20

(a)

0

50 z

x20– 20

(b)

0

cha9792x_ch28_811-840.indd Page 821 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 821 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

822 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

These points are called strange attractors in the jargon of mathematicians who study such nonlinear systems. Solutions such as the type we have explored for the Lorenz equations are referred to as chaotic solutions. The study of chaos and nonlinear systems presently represents an exciting area of analysis that has implications to mathematics as well as to science and engineering. From a numerical perspective, the primary point is the sensitivity of such solutions to initial conditions. Thus, different numerical algorithms, computer precision, and inte- gration time steps can all have an impact on the resulting numerical solution.

28.3 SIMULATING TRANSIENT CURRENT FOR AN ELECTRIC CIRCUIT (ELECTRICAL ENGINEERING)

Background. Electric circuits where the current is time-variable rather than constant are common. A transient current is established in the right-hand loop of the circuit shown in Fig. 28.11 when the switch is suddenly closed. Equations that describe the transient behavior of the circuit in Fig. 28.11 are based on Kirchhoff’s law, which states that the algebraic sum of the voltage drops around a closed loop is zero (recall Sec. 8.3). Thus,

L di

dt 1 Ri 1

q

C 2 E(t) 5 0 (28.9)

where L(diydt) 5 voltage drop across the inductor, L 5 inductance (H), R 5 resistance (V), q 5 charge on the capacitor (C), C 5 capacitance (F), E(t) 5 time-variable voltage source (V), and

i 5 dq

dt (28.10)

Equations (28.9) and (28.10) are a pair of fi rst-order linear differential equations that can be solved analytically. For example, if E(t) 5 E0 sin vt and R 5 0,

q(t) 5 2E0

L( p2 2 v2) v

p sin pt 1

E0 L(p2 2 v2)

sin vt (28.11)

FIGURE 28.11 An electric circuit where the current varies with time.

Switch

Resistor

Capacitor –

+ V0

E(t)

+ Battery Inductor

cha9792x_ch28_811-840.indd Page 822 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 822 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.3 SIMULATING TRANSIENT CURRENT FOR AN ELECTRIC CIRCUIT 823

where p 5 1y1LC. The values of q and dqydt are zero for t 5 0. Use a numerical ap- proach to solve Eqs. (28.9) and (28.10) and compare the results with Eq. (28.11).

Solution. This problem involves a rather long integration range and demands the use of a highly accurate scheme to solve the differential equation if good results are expected. Let us assume that L 5 1 H, E0 5 1V, C 5 0.25 C, and v

2 5 3.5 s2. This gives p 5 2, and Eq. (28.11) becomes

q(t) 5 21.8708 sin (2t) 1 2 sin (1.8708t)

for the analytical solution. This function is plotted in Fig. 28.12. The rapidly chang- ing nature of the function places a severe requirement on any numerical procedure to fi nd q(t). Furthermore, because the function exhibits a slowly varying periodic nature as well as a rapidly varying component, long integration ranges are necessary to portray the solution. Thus, we expect that a high-order method is preferred for this problem. However, we can try both Euler and fourth-order RK methods and compare the results. Using a step size of 0.1 s gives a value for q at t 5 10 s of 26.638 with Euler’s method and a value of 21.9897 with the fourth-order RK method. These results compare to an exact solution of 21.996 C. Figure 28.13 shows the results of Euler integration every 1.0 s compared to the exact solution. Note that only every tenth output point is plotted. It is seen that the global error increases as t increases. This divergent behavior intensifi es as t approaches infi nity. In addition to directly simulating a network’s transient response, numerical methods can also be used to determine its eigenvalues. For example, Fig. 28.14 shows an LC network for which Kirchhoff’s voltage law can be employed to develop the following

FIGURE 28.12 Computer screen showing the plot of the function represented by Eq. (28.11).

– 6.0

6.0

0

4.0

– 4.0

– 2.0

2.0

0 40 Time

C u

rr en

t

Capacitor

20 8060 100

cha9792x_ch28_811-840.indd Page 823 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 823 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

824 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

system of ODEs:

2L1 di1 dt

2 1

C1 #

t

2q (i1 2 i2) dt 5 0

2L2 di2 dt

2 1

C2 #

t

2q (i2 2 i3) dt 1

1

C1 #

t

2q (i1 2 i2) dt 5 0

2L3 di3 dt

2 1

C3 #

t

2q i3 dt 1

1

C2 #

t

2q (i2 2 i3) dt 5 0

Notice that we have represented the voltage drop across the capacitor as

VC 5 1

C # t

2q i dt

This is an alternative and equivalent expression to the relationship used in Eq. (28.9) and introduced in Sec. 8.3.

FIGURE 28.13 Results of Euler integration versus exact solution. Note that only every tenth output point is plotted.

0

2

4

Charge

– 6

– 4

– 2

t, s10

Euler’s method

FIGURE 28.14 An LC network.

L1 L2 L3

C1 i1

+

C2 C3 i2 i3

cha9792x_ch28_811-840.indd Page 824 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 824 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.3 SIMULATING TRANSIENT CURRENT FOR AN ELECTRIC CIRCUIT 825

The system of ODEs can be differentiated and rearranged to yield

L1 d 2i1 dt 2

1 1

C1 (i1 2 i2) 5 0

L2 d 2i2 dt 2

1 1

C 2 (i2 2 i3) 2

1

C1 (i1 2 i2) 5 0

L3 d 2i3 dt2

1 1

C3 i3 2

1

C2 (i2 2 i3) 5 0

Comparison of this system with the one in Eq. (27.5) indicates an analogy between a spring-mass system and an LC circuit. As was done with Eq. (27.5), the solution can be assumed to be of the form

ij 5 Aj sin(vt)

This solution along with its second derivative can be substituted into the simultaneous ODEs. After simplifi cation, the result is

a 1 C1

2 L1v 2b A1 2 1C2 A2 5 0

2 1

C1 A1 1 a 1C1 1

1

C2 2 L2v

2b A2 2 1C2 A3 5 0 2

1

C2 A2 1 a 1C2 1

1

C3 2 L3v

2b A3 5 0 Thus, we have formulated an eigenvalue problem. Further simplifi cation results for the special case where the C’s and L’s are constant. For this situation, the system can be expressed in matrix form as

£ 1 2 l 21 0

21 2 2 l 21

0 21 2 2 l

§ • A1 A2 A3

¶ 5 {0} (28.12)

where

l 5 LCv2 (28.13)

Numerical methods can be employed to determine values for the eigenvalues and eigenvectors. MATLAB is particularly convenient in this regard. The following MATLAB session has been developed to do this:

>>a=[1 −1 0; −1 2 −1; 0 21 2]

a =

1 −1 0 −1 2 −1 0 −1 2

cha9792x_ch28_811-840.indd Page 825 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 825 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

826 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

>>[v,d]=eig(a)

v =

0.7370 0.5910 0.3280 0.5910 −0.3280 −0.7370 0.3280 −0.7370 0.5910

d =

0.1981 0 0 0 1.5550 0 0 0 3.2470

The matrix v consists of the system’s three eigenvectors (arranged as columns), and d is a matrix with the corresponding eigenvalues on the diagonal. Thus, the package com- putes that the eigenvalues are l 5 0.1981, 1.555, and 3.247. These values in turn can be substituted into Eq. (28.13) to solve for the natural circular frequencies of the system

v 5 f 0.44501LC1.2470 1LC

1.8019

1LC

Aside from providing the natural frequencies, the eigenvalues can be substituted into Eq. (28.12) to gain further insight into the circuit’s physical behavior. For example, substituting l 5 0.1981 yields

£ 0.8019 21 0

21 1.8019 21

0 21 1.8019

§ • i1 i2 i3

¶ 5 {0}

Although this system does not have a unique solution, it will be satisfi ed if the currents are in fi xed ratios, as in

0.8019i1 5 i2 5 1.8019i3 (28.14)

Thus, as depicted in Fig. 28.15a, they oscillate in the same direction with different mag- nitudes. Observe that if we assume that i1 5 0.737, we can use Eq. (28.14) to compute the other currents with the result

{i} 5 • 0.737

0.591

0.328

which is the fi rst column of the v matrix calculated with MATLAB.

cha9792x_ch28_811-840.indd Page 826 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 826 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.4 THE SWINGING PENDULUM 827

In a similar fashion, the second eigenvalue of l 5 1.555 can be substituted and the result evaluated to yield

21.8018i1 5 i2 5 2.247i3

As depicted in Fig. 28.15b, the fi rst loop oscillates in the opposite direction from the second and third. Finally, the third mode can be determined as

20.445i1 5 i2 5 20.8718i3

Consequently, as in Fig. 28.15c, the fi rst and third loops oscillate in the opposite direction from the second.

28.4 THE SWINGING PENDULUM (MECHANICAL/AEROSPACE ENGINEERING)

Background. Mechanical engineers (as well as all other engineers) are frequently faced with problems concerning the periodic motion of free bodies. The engineering approach to such problems ultimately requires that the position and velocity of the body be known as a function of time. These functions of time invariably are the solution of ordinary differential equations. The differential equations are usually based on Newton’s laws of motion. As an example, consider the simple pendulum shown previously in Fig. PT7.1. The particle of weight W is suspended on a weightless rod of length l. The only forces acting on the particle are its weight and the tension R in the rod. The position of the particle at any time is completely specifi ed in terms of the angle u and l. The free-body diagram in Fig. 28.16 shows the forces on the particle and the acceleration. It is convenient to apply Newton’s laws of motion in the x direction tangent to the path of the particle:

gF 5 2W sin u 5 W g

a

where g 5 the gravitational constant (32.2 ft/s2) and a 5 the acceleration in the x direction. The angular acceleration of the particle (a) becomes

a 5 a

l

(a) � = 0.4451 LC

(b) � = 1.2470 LC

(c) � = 1.8019 LC

FIGURE 28.15 A visual representation of the natural modes of oscillation of the LC network of Fig. 28.14. Note that the diameters of the circular arrows are proportional to the magnitudes of the currents for each loop.

FIGURE 28.16 A free-body diagram of the swinging pendulum showing the forces on the particle and the acceleration.

W y

a

x

R

cha9792x_ch28_811-840.indd Page 827 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 827 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

828 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

Therefore, in polar coordinates (a 5 d2uydt2),

2W sin u 5 Wl g

a 5 Wl g

d 2u

dt 2

or

d 2u

dt2 1

g

l sin u 5 0 (28.15)

This apparently simple equation is a second-order nonlinear differential equation. In general, such equations are diffi cult or impossible to solve analytically. You have two choices regarding further progress. First, the differential equation might be reduced to a form that can be solved analytically (recall Sec. PT7.1.1), or second, a numerical approximation technique can be used to solve the differential equation directly. We will examine both of these alternatives in this example.

Solution. Proceeding with the fi rst approach, we note that the series expansion for sin u is given by

sin u 5 u 2 u3

3! 1 u5

5! 2 u7

7! 1 p (28.16)

For small angular displacements, sin u is approximately equal to u when expressed in radians. Therefore, for small displacements, Eq. (28.15) becomes

d 2u

dt2 1

g

l u 5 0 (28.17)

which is a second-order linear differential equation. This approximation is very important because Eq. (28.17) is easy to solve analytically. The solution, based on the theory of differential equations, is given by

u(t) 5 u0 cos B

g

l t (28.18)

where u0 5 the displacement at t 5 0 and where it is assumed that the velocity (y 5 duydt) is zero at t 5 0. The time required for the pendulum to complete one cycle of oscillation is called the period and is given by

T 5 2p  B

l g

(28.19)

Figure 28.17 shows a plot of the displacement u and velocity duydt as a function of time, as calculated from Eq. (28.18) with u0 5 py4 and l 5 2 ft. The period, as calculated from Eq. (28.19), is 1.5659 s. The above calculations essentially are a complete solution of the motion of the pendulum. However, you must also consider the accuracy of the results because of the

cha9792x_ch28_811-840.indd Page 828 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 828 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

28.4 THE SWINGING PENDULUM 829

assumptions inherent in Eq. (28.17). To evaluate the accuracy, it is necessary to obtain a numerical solution for Eq. (28.15), which is a more complete physical representation of the motion. Any of the methods discussed in Chaps. 25 and 26 could be used for this purpose—for example, the Euler and fourth-order RK methods. Equation (28.15) must be transformed into two fi rst-order equations to be compatible with the above methods. This is accomplished as follows. The velocity y is defi ned by

du

dt 5 y (28.20)

and, therefore, Eq. (28.15) can be expressed as

dy

dt 5 2

g

l sin u (28.21)

Equations (28.20) and (28.21) are a coupled system of two ordinary differential equa- tions. The numerical solutions by the Euler method and the fourth-order RK method give the results shown in Table 28.1, which compares the analytic solution for the linear equation of motion [Eq. (28.18)] in column (a) with the numerical solutions in columns (b), (c), and (d). The Euler and fourth-order RK methods yield different results and both disagree with the analytic solution, although the fourth-order RK method for the nonlinear case is closer to the analytic solution than is the Euler method. To properly evaluate the difference between the linear and nonlinear models, it is important to determine the accuracy of the numerical results. This is accomplished in three ways. First, the Euler numerical solution is easily recognized as inadequate because it overshoots the initial condition at t 5 0.8 s. This clearly violates conservation of energy. Second, column (c) and (d) in Table 28.1 show the solution of the fourth-order RK method

– 0.8

0.8

0 t

– 2

2

0 t

d� dt

FIGURE 28.17 Plot of displacement u and velocity du/dt as a function of time t, as calculated from Eq. (28.18). u0 is p/4 and the length is 2 ft.

cha9792x_ch28_811-840.indd Page 829 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 829 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

830 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

TABLE 28.1 Comparison of a linear analytical solution of the swinging pendulum problem with three nonlinear numerical solutions.

Nonlinear Numerical Solutions

Linear Analytical Euler 4th-Order RK 4th-Order RK Time, Solution (h 5 0.05) (h 5 0.05) (h 5 0.01) s (a) (b) (c) (d)

0.0 0.785398 0.785398 0.785398 0.785398 0.2 0.545784 0.615453 0.566582 0.566579 0.4 20.026852 0.050228 0.021895 0.021882 0.6 20.583104 20.639652 20.535802 20.535820 0.8 20.783562 21.050679 20.784236 20.784242 1.0 20.505912 20.940622 20.595598 20.595583 1.2 0.080431 20.299819 20.065611 20.065575 1.4 0.617698 0.621700 0.503352 0.503392 1.6 0.778062 1.316795 0.780762 0.780777

for step sizes of 0.05 and 0.01. Because these vary in the fourth decimal place, it is reasonable to assume that the solution with a step size of 0.01 is also accurate with this degree of certainty. Third, for the 0.01-s step-size case, u obtains a local maxi- mum value of 0.785385 at t 5 1.63 s (not shown in Table 28.1). This indicates that the pendulum returns to its original position with four-place accuracy with a period of 1.63 s. These considerations allow you to safely assume that the difference be- tween columns (a) and (d) in Table 28.1 truly represents the difference between the linear and nonlinear model. Another way to characterize the difference between the linear and the nonlinear model is on the basis of period. Table 28.2 shows the period of oscillation as calculated by the linear model and nonlinear model for three different initial displacements. It is seen that the calculated periods agree closely when u is small because u is a good approximation for sin u in Eq. (28.16). This approximation deteriorates when u becomes large. These analyses are typical of cases you will routinely encounter as an engineer. The utility of the numerical techniques becomes particularly signifi cant in nonlinear prob- lems, and in many cases real-world problems are nonlinear.

TABLE 28.2 Comparison of the period of an oscillating body calculated from linear and nonlinear models.

Period, s

Initial Linear Model Nonlinear Model Displacement, U0 (T 5 2P1Iyg) [Numerical Solution of Eq. (28.15)]

p/16 1.5659 1.57 p/4 1.5659 1.63 p/2 1.5659 1.85

cha9792x_ch28_811-840.indd Page 830 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 830 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 831

PROBLEMS

Chemical/Bio Engineering 28.1 Perform the fi rst computation in Sec. 28.1, but for the case where h 5 10. Use the Heun (without iteration) and the fourth- order RK method to obtain solutions. 28.2 Perform the second computation in Sec. 28.1, but for the system described in Prob. 12.4. 28.3 A mass balance for a chemical in a completely mixed reactor can be written as

V dc

dt 5 F 2 Qc 2 kVc2

where V 5 volume (12 m3), c 5 concentration (g/m3), F 5 feed rate (175 g/min), Q 5 fl ow rate (1 m3/min), and k 5 a second-order reaction rate (0.15 m3/g/min). If c(0) 5 0, solve the ODE until the concentration reaches a stable level. Use the midpoint method (h 5 0.5) and plot your results.

Challenge question: If one ignores the fact that concentrations must be positive, fi nd a range of initial conditions such that you obtain a very different trajectory than was obtained with c(0) 5 0. Relate your results to the steady-state solutions. 28.4 If cin 5 cb(1 2 e

20.12t), calculate the outfl ow concentration of a conservative substance (no reaction) for a single, completely mixed reactor as a function of time. Use Heun’s method (without itera- tion) to perform the computation. Employ values of cb 5 40 mg/m

3, Q 5 6 m3/min, V 5 100 m3, and c0 5 20 mg/m

3. Perform the com- putation from t 5 0 to 100 min using h 5 2. Plot your results along with the infl ow concentration versus time. 28.5 Seawater with a concentration of 8000 g/m3 is pumped into a well-mixed tank at a rate of 0.6 m3/hr. Because of faulty design work, water is evaporating from the tank at a rate of 0.025 m3/hr. The salt solution leaves the tank at a rate of 0.6 m3/hr. (a) If the tank originally contains 1 m3 of the inlet solution, how

long after the outlet pump is turned on will the tank run dry? (b) Use numerical methods to determine the salt concentration in

the tank as a function of time. 28.6 A spherical ice cube (an “ice sphere”) that is 6 cm in diam- eter is removed from a 08C freezer and placed on a mesh screen at room temperature Ta 5 208C. What will be the diameter of the ice cube as a function of time out of the freezer (assuming that all the water that has melted immediately drips through the screen)? The heat transfer coeffi cient h for a sphere in a still room is about 3 W/(m2 ? K). The heat fl ux from the ice sphere to the air is given by

Flux 5 q

A 5 h(Ta 2 T)

where q 5 heat and A 5 surface area of the sphere. Use a numerical method to make your calculation. Note that the latent heat of fusion is 333 kJ/kg and the density of ice is approximately 0.917 kg/m3. 28.7 The following equations defi ne the concentrations of three reactants:

dca dt

5 210cacc 1 cb

dcb dt

5 10cacc 2 cb

dcc dt

5 210cacc 1 cb 2 2cc

If the initial conditions are ca 5 50, cb 5 0, and cc 5 40, fi nd the concentrations for the times from 0 to 3 s. 28.8 Compound A diffuses through a 4-cm-long tube and reacts as it diffuses. The equation governing diffusion with reaction is

D d 2A

dx2 2 kA 5 0

At one end of the tube, there is a large source of A at a concentra- tion of 0.1 M. At the other end of the tube there is an adsorbent material that quickly absorbs any A, making the concentration 0 M. If D 5 1.5 3 1026 cm2/s and k 5 5 3 1026 s21, what is the concen- tration of A as a function of distance in the tube? 28.9 In the investigation of a homicide or accidental death, it is often important to estimate the time of death. From the experimen- tal observations, it is known that the surface temperature of an object changes at a rate proportional to the difference between the temperature of the object and that of the surrounding environment or ambient temperature. This is known as Newton’s law of cooling. Thus, if T(t) is the temperature of the object at time t, and Ta is the constant ambient temperature:

dT

dt 5 2K(T 2 Ta)

where K . 0 is a constant of proportionality. Suppose that at time t 5 0 a corpse is discovered and its temperature is measured to be To. We assume that at the time of death, the body temperature, Td, was at the normal value of 378C. Suppose that the temperature of the corpse when it was discovered was 29.58C, and that two hours later, it is 23.58C. The ambient temperature is 208C. (a) Determine K and the time of death. (b) Solve the ODE numerically and plot the results. 28.10 The reaction AS B takes place in two reactors in series. The reactors are well mixed but are not at steady state. The

cha9792x_ch28_811-840.indd Page 831 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 831 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

832 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

L (cm), a chemical compound A diffuses into the biofi lm where it is subject to an irreversible fi rst-order reaction that converts it to a product B.

Steady-state mass balances can be used to derive the following ordinary differential equations for compound A:

D d 2ca dx2

5 0  0 # x , L

Df d 2ca dx2

2 kca 5 0  L # x , L 1 Lf

where D 5 the diffusion coeffi cient in the diffusion layer 5 0.8 cm2/d, Df 5 the diffusion coeffi cient in the biofi lm 5 0.64 cm

2/d, and k 5 the fi rst-order rate for the conversion of A to B 5 0.1/d. The following boundary conditions hold:

ca 5 ca0  at x 5 0 dca dx

5 0  at x 5 L 1 Lf

where ca 0 5 the concentration of A in the bulk liquid 5 100 mol/L. Use the fi nite-difference method to compute the steady-state distri- bution of A from x 5 0 to L 1 Lf, where L 5 0.008 cm and Lf 5 0.004 cm. Employ centered fi nite differences with Dx 5 0.001 cm. 28.14 The following differential equation describes the steady- state concentration of a substance that reacts with fi rst-order kinet- ics in an axially-dispersed plug-fl ow reactor (Fig. P28.14),

D d 2c

dx2 2 U

dc

dx 2 kc 5 0

unsteady-state mass balance for each stirred tank reactor is shown below:

dCA1 dt

5 1 t

(CA0 2 CA1) 2 kCA1

dCB1 dt

5 2 1 t

CB1 1 kCA1

dCA2 dt

5 1 t

(CA1 2 CA2) 2 kCA2

dCB2 dt

5 1 t

(CB1 2 CB2) 1 kCA2

where CA0 5 concentration of A at the inlet of the fi rst reactor, CA1 5 concentration of A at the outlet of the fi rst reactor (and inlet of the second), CA2 5 concentration of A at the outlet of the second reac- tor, CB1 5 concentration of B at the outlet of the fi rst reactor (and inlet of the second), CB2 5 concentration of B in the second reactor, t 5 residence time for each reactor, and k 5 the rate constant for reaction of A to produce B. If CA0 is equal to 20, fi nd the concentra- tions of A and B in both reactors during their fi rst 10 minutes of operation. Use k 5 0.12/min and t 5 5 min and assume that the initial conditions of all the dependent variables are zero. 28.11 A nonisothermal batch reactor can be described by the following equations:

dC

dt 5 2e(210y(T1273))C

dT

dt 5 1000e(210y(T1273))C 2 10(T 2 20)

where C is the concentration of the reactant and T is the tempera- ture of the reactor. Initially the reactor is at 158C and has a concen- tration of reactant C of 1.0 gmol/L. Find the concentration and temperature of the reactor as a function of time. 28.12 The following system is a classic example of stiff ODEs that can occur in the solution of chemical reaction kinetics:

dc1 dt

5 20.013c1 2 1000c1c3

dc2 dt

5 22500c2c3

dc3 dt

5 20.013c1 2 1000c1c3 2 2500c2c3

Solve these equations from t 5 0 to 50 with initial conditions c1(0) 5 c2(0) 5 1 and c3(0) 5 0. If you have access to MATLAB software, use both standard (for example, ode45) and stiff (for example, ode23s) functions to obtain your solutions. 28.13 A biofi lm with a thickness Lf (cm) grows on the surface of a solid (Fig. P28.13). After traversing a diffusion layer of thickness

Bulk liquid

0

Diffusion layer Biofilm

x

Solid surface

L Lf

FIGURE P28.13 A biofi lm growing on a solid surface.

cha9792x_ch28_811-840.indd Page 832 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 832 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 833

Use the fi nite-difference approach to solve for the concentration of each reactant as a function of distance given: D 5 0.1 m2/min, U 5 1 m/min, k1 5 3/min, k2 5 1/min, L 5 0.5 m, ca,in 5 10 mol/L. Employ centered fi nite-difference approximations with Dx 5 0.05 m to obtain your solutions and assume Danckwerts boundary condi- tions, as described in Prob. 28.14. Also, compute the sum of the reactants as a function of distance. Do your results make sense? 28.16 Bacteria growing in a batch reactor utilize a soluble food source (substrate) as depicted in Fig. P28.16. The uptake of the substrate is represented by a logistic model with Michaelis-Menten limitation. Death of the bacteria produces detritus which is subse- quently converted to the substrate by hydrolysis. In addition, the bacteria also excrete some substrate directly. Death, hydrolysis, and excretion are all simulated as fi rst-order reactions.

Mass balances can be written as

dX

dt 5 mmax a1 2 XKb a

S

Ks 1 S b X 2 kd X 2 ke X

dC

dt 5 kd X 2 khC

dS

dt 5 ke X 1 khC 2 mmax a1 2 XKb a

S

Ks 1 S b X

where X, C, and S 5 the concentrations (mg/L) of bacteria, detritus, and substrate, respectively; mmax 5 maximum growth rate (/d), K 5 the logistic carrying capacity (mg/L); Ks 5 the Michaelis-Menten half-saturation constant (mg/L), kd 5 death rate (/d); ke 5 excretion rate (/d); and kh 5 hydrolysis rate (/d). Simulate the concentrations from t 5 0 to 100 d, given the initial conditions X(0) 5 1 mg/L, S(0) 5 100 mg/L, and C(0) 5 0 mg/L. Employ the following param- eters in your calculation: mmax 5 10/d, K 5 10 mg/L, Ks 5 10 mg/L, kd 5 0.1/d, ke 5 0.1/d, and kh 5 0.1/d.

Civil/Environmental Engineering 28.17 Perform the same computation for the Lotka-Volterra sys- tem in Sec. 28.2, but use (a) Euler’s method, (b) Heun’s method (without iterating the corrector), (c) the fourth-order RK method, and (d) the MATLAB ode45 function. In all cases use single- precision variables, a step size of 0.1, and simulate from t 5 0 to 20. Develop phase-plane plots for all cases.

where D 5 the dispersion coeffi cient (m2/hr), c 5 concentration (mol/L), x 5 distance (m), U 5 the velocity (m/hr), and k 5 the reaction rate (/hr). The boundary conditions can be formulated as

Ucin 5 Uc(x 5 0) 2 D dc

dx (x 5 0)

dc

dx (x 5 L) 5 0

where cin 5 the concentration in the infl ow (mol/L), and L 5 the length of the reactor (m). These are called Danckwerts boundary conditions. Use the fi nite-difference approach to solve for concen- tration as a function of distance given the following parameters: D 5 5000 m2/hr, U 5 100 m/hr, k 5 2/hr, L 5 100 m, and cin 5 100 mol/L. Employ centered fi nite-difference approximations with Dx 5 10 m to obtain your solutions. Compare your numerical results with the analytical solution,

c 5 Ucin

(U 2 Dl1)l2e l2L 2 (U 2 Dl2)l1e

l1L

3 (l2e l2Lel1x 2 l1e

l1Lel2x)

where

l1

l2 5

U

2D a1 ;

A 1 1

4k D

U2 b

28.15 A series of fi rst-order, liquid-phase reactions create a desir- able product (B) and an undesirable byproduct (C)

AS k1

BS k2

C

If the reactions take place in an axially-dispersed plug-fl ow reactor (Fig. P28.14), steady-state mass balances can be used to develop the following second-order ODEs,

D d 2ca dx2

2 U dca dx

2 k1ca 5 0

D d 2cb dx2

2 U dcb dx

1 k1ca 2 k2cb 5 0

D d 2cc dx2

2 U dcc dx

1 k2cb 5 0

FIGURE P28.14 An axially-dispersed plug-fl ow reactor.

x � 0 x � L x

hydrolysis

excretion

deathuptakeSubstrate S

Bacteria X

Detritus C

FIGURE P28.16

cha9792x_ch28_811-840.indd Page 833 06/11/13 8:21 PM F-468 cha9792x_ch28_811-840.indd Page 833 06/11/13 8:21 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

834 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

Using mass balances, the system can be modeled as the following simultaneous ODEs:

V1 dc1 dt

5 2Qc1 1 E12(c2 2 c1) 1 E13(c3 2 c1)

V2 dc2 dt

5 E12(c1 2 c2)

V3 dc3 dt

5 E13(c1 2 c3)

where Vi 5 volume of segment i, Q 5 fl ow, and Eij 5 diffusive mixing rate between segments i and j. Use the data and the differ- ential equations to estimate the E’s if V1 5 1 3 10

7, V2 5 8 3 10 6,

V3 5 5 3 10 6, and Q 5 4 3 106. Employ Euler’s method with a step

size of 0.1 for your analysis. 28.22 Population-growth dynamics are important in a variety of planning studies for areas such as transportation and water-resource engineering. One of the simplest models of such growth incorpo- rates the assumption that the rate of change of the population p is proportional to the existing population at any time t:

dp

dt 5 Gp (P28.22.1)

where G 5 a growth rate (per year). This model makes intuitive sense because the greater the population, the greater the number of poten- tial parents. At time t 5 0, an island has a population of 6000 people. If G 5 0.075 per year, employ Heun’s method (without iteration) to predict the population at t 5 20 years, using a step size of 0.5 year. Plot p versus t on standard and semilog graph paper. Determine the slope of the line on the semilog plot. Discuss your results. 28.23 Although the model in Prob. 28.22 works adequately when population growth is unlimited, it breaks down when factors such as food shortages, pollution, and lack of space inhibit growth. In such cases, the growth rate itself can be thought of as being inversely proportional to population. One model of this relationship is

G 5 G¿(pmax 2 p) (P28.23.1)

where G9 5 a population-dependent growth rate (per people-year) and pmax 5 the maximum sustainable population. Thus, when popula- tion is small (p V pmax), the growth rate will be at a high constant rate of G9 pmax. For such cases, growth is unlimited and Eq. (P28.23.1) is essentially identical to Eq. (P28.22.1). However, as population grows (that is, p approaches pmax), G decreases until at p 5 pmax it is zero. Thus, the model predicts that, when the population reaches the maximum sustainable level, growth is nonexistent, and the system is at a steady state. Substituting Eq. (P28.23.1) into Eq. (P28.22.1) yields

dp

dt 5 G¿(pmax 2 p)p

28.18 Perform the same computation for the Lorenz equations in Sec. 28.2, but use (a) Euler’s method, (b) Heun’s method (without iterating the corrector), (c) the fourth-order RK method, and (d) the MATLAB ode45 function. In all cases use single-precision vari- ables and a step size of 0.1 and simulate from t 5 0 to 20. Develop phase-plane plots for all cases. 28.19 The following equation can be used to model the defl ection of a sailboat mast subject to a wind force:

d 2y

dz2 5

f

2EI (L 2 z)2

where f 5 wind force, E 5 modulus of elasticity, L 5 mast length, and I 5 moment of inertia. Calculate the defl ection if y 5 0 and dyydz 5 0 at z 5 0. Use parameter values of f 5 60, L 5 30, E 5 1.25 3 108, and I 5 0.05 for your computation. 28.20 Perform the same computation as in Prob. 28.19, but rather than using a constant wind force, employ a force that varies with height according to (recall Sec. 24.2)

f (z) 5 200z

5 1 z e22zy30

28.21 An environmental engineer is interested in estimating the mixing that occurs between a stratifi ed lake and an adjacent em- bayment (Fig. P28.21). A conservative tracer is instantaneously mixed with the bay water, and then the tracer concentration is monitored over the ensuing period in all three segments. The val- ues are

t 0 2 4 6 8 12 16 20 c1 0 15 11 7 6 3 2 1 c2 0 3 5 7 7 6 4 2 c3 100 48 26 16 10 4 3 2

FIGURE P28.21

Bay (3)

Upper layer (1)

Lower layer (2)

cha9792x_ch28_811-840.indd Page 834 06/11/13 8:22 PM F-468 cha9792x_ch28_811-840.indd Page 834 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 835

tension in the cable is a minimum of To. The differential equation that governs the cable is

d 2y

dx2 5

wo To c 1 1 sin apx

2lA b d

Solve this equation using a numerical method and plot the shape of the cable (y versus x). For the numerical solution, the value of To is unknown, so the solution must use an iterative technique, similar to the shooting method, to converge on a correct value of hA for vari- ous values of To. 28.26 The basic differential equation of the elastic curve for a can- tilever beam (Fig. P28.26) is given as

EI d 2y

dx2 5 2P(L 2 x)

where E 5 the modulus of elasticity and I 5 the moment of inertia. Solve for the defl ection of the beam using a numerical method. The following parameter values apply: E 5 30,000 ksi, I 5 800 in4,

For the same island studied in Prob. 28.22, employ Heun’s method (without iteration) to predict the population at t 5 20 years, using a step size of 0.5 year. Employ values of G9 5 1025 per people-year and pmax 5 20,000 people. At time t 5 0, the island has a population of 6000 people. Plot p versus t and interpret the shape of the curve. 28.24 Isle Royale National Park is a 210-square-mile archipelago composed of a single large island and many small islands in Lake Superior. Moose arrived around 1900 and by 1930, their population approached 3000, ravaging vegetation. In 1949, wolves crossed an ice bridge from Ontario. Since the late 1950s, the numbers of the moose and wolves have been tracked. (Dash indicates no data.)

Year Moose Wolves Year Moose Wolves

1960 700 22 1972 836 23 1961 — 22 1973 802 24 1962 — 23 1974 815 30 1963 — 20 1975 778 41 1964 — 25 1976 641 43 1965 — 28 1977 507 33 1966 881 24 1978 543 40 1967 — 22 1979 675 42 1968 1000 22 1980 577 50 1969 1150 17 1981 570 30 1970 966 18 1982 590 13 1971 674 20 1983 811 23

(a) Integrate the Lotka-Volterra equations from 1960 through 2020. Determine the coeffi cient values that yield an optimal fi t. Compare your simulation with these data using a time-series approach, and comment on the results.

(b) Plot the simulation of (a), but use a phase-plane approach. (c) After 1993, suppose that the wildlife managers trap one wolf per

year and transport it off the island. Predict how the populations of both the wolves and moose would evolve to the year 2020. Present your results as both time-series and phase-plane plots. For this case, as well as for (d), use the following coeffi cients: a 5 0.3, b 5 0.01111, c 5 0.2106, d 5 0.0002632.

(d) Suppose that in 1993, some poachers snuck onto the island and killed 50% of the moose. Predict how the populations of both the wolves and moose would evolve to the year 2020. Present your results as both time-series and phase-plane plots.

28.25 A cable is hanging from two supports at A and B (Fig. P28.25). The cable is loaded with a distributed load whose magnitude varies with x as

w 5 wo c 1 1 sin apx2lAb d where wo 5 1000 lb/ft. The slope of the cable (dyydx) 5 0 at x 5 0, which is the lowest point for the cable. It is also the point where the

FIGURE P28.25

w = wo[1 + sin (�x/2la)]

lA = 200 ft

x B

A

y

hA = 50 ft

FIGURE P28.26

L P

y

x

0

cha9792x_ch28_811-840.indd Page 835 06/11/13 8:22 PM F-468 cha9792x_ch28_811-840.indd Page 835 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

836 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

where h 5 depth (m), t 5 time (s), d 5 pipe diameter (m), A(h) 5 pond surface area as a function of depth (m2), g 5 gravitational constant (5 9.81 m/s2), and e 5 depth of pipe outlet below the pond bottom (m). Based on the following area-depth table, solve this dif- ferential equation to determine how long it takes for the pond to empty given that h(0) 5 6 m, d 5 0.25 m, e 5 1 m.

h, m 6 5 4 3 2 1 0

A(h), 104 m2 1.17 0.97 0.67 0.45 0.32 0.18 0

28.29 Engineers and scientists use mass-spring models to gain in- sight into the dynamics of structures under the infl uence of distur- bances such as earthquakes. Figure P28.29 shows such a representation for a three-story building. For this case, the analysis is limited to horizontal motion of the structure. Force balances can be developed for this system as

ak1 1 k2 m1

2 v2b X1  2 k2m1  X2   5 0

2 k2 m2

 X1 1 ak2 1 k3m2 2 v 2b X2 2 k3m2 X3 5 0

2 k3 m3

 X2 1 a k3m3 2 v 2b X3 5 0

Determine the eigenvalues and eigenvectors and graphically repre- sent the modes of vibration for the structure by displaying the amplitudes versus height for each of the eigenvectors. Normalize the amplitudes so that the displacement of the third fl oor is one. 28.30 Under a number of simplifying assumptions, the steady- state height of the water table in a one-dimensional, unconfi ned groundwater aquifer (Fig. P28.30) can be modeled with the follow- ing second-order ODE,

K h d 2h

dx2 1 N 5 0

P 5 1 kip, L 5 10 ft. Compare your numerical results to the analytical solution,

y 5 2 PLx2

2EI 1

Px3

6EI

28.27 The basic differential equation of the elastic curve for a uni- formly loaded beam (Fig. P28.27) is given as

EI d 2y

dx2 5

wLx

2 2

wx2

2

where E 5 the modulus of elasticity and I 5 the moment of inertia. Solve for the defl ection of the beam using (a) the fi nite-difference approach (Dx 5 2 ft) and (b) the shooting method. The following parameter values apply: E 5 30,000 ksi, I 5 800 in4, w 5 1 kip/ft, L 5 10 ft. Compare your numerical results to the analytical solution,

y 5 wLx3

12EI 2

wx4

24EI 2

wL3x

24EI

28.28 A pond drains through a pipe, as shown in Fig. P28.28. Un- der a number of simplifying assumptions, the following differential equation describes how depth changes with time:

dh

dt 5 2

pd 2

4A(h) 12g(h 1 e)

L

y

w

x

0

FIGURE P28.27

m3 = 8000 kg

k3 = 1800 kN/m

k2 = 2400 kN/m

k1 = 3000 kN/m

m2 = 10,000 kg

m1 = 12,000 kg

FIGURE P28.29

e

d

h

A(h)

FIGURE P28.28

cha9792x_ch28_811-840.indd Page 836 06/11/13 8:22 PM F-468 cha9792x_ch28_811-840.indd Page 836 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 837

dy

dt 5 2cy 1 dxy

where K 5 the carrying capacity. Use the same parameter values and initial conditions as in Sec. 28.2 to integrate these equations from t 5 0 to 100 using ode45. (a) Employ a very large value of K 5 108 to validate that you ob-

tain the same results as in Sec. 28.2. (b) Compare (a) with the more realistic carrying capacity of K 5

200. Discuss your results. 28.33 The growth of fl oating, unicellular algae below a sewage treatment plant discharge can be modeled with the following simul- taneous ODEs:

da

dt 5 [kg(n, p) 2 kd 2 ks] a

dn

dt 5 rnckhc 2 rnakg(n, p)a

dp

dt 5 rpckhc 2 rpakg(n, p)a

dc

dt 5 rcakda 2 khc

where t 5 travel time (d), a 5 algal chlorophyll concentration (mgA/L), n 5 inorganic nitrogen concentration (mgN/L), p 5 inor- ganic phosphorus concentration (mgP/L), c 5 detritus concentra- tion (mgC/L), kd 5 algal death rate (/d), ks 5 algal settling rate (/d), kh 5 detrital hydrolysis rate (/d), rnc 5 nitrogen-to-carbon ratio (mgN/mgC), rpc 5 phosphorus-to-carbon ratio (mgP/mgC), rna 5 nitrogen-to-chlorophyll ratio (mgN/mgA), rpa 5 phosphorus-to- chlorophyll ratio (mgP/mgA), and kg(n, p) 5 algal growth rate (/d), which can be computed with

kg(n, p) 5 kg min e pksp 1 p, n

ksn 1 n f

where kg 5 the algal growth rate at excess nutrient levels (/d), ksp 5 the phosphorus half-saturation constant (mgP/L), and ksn 5 the nitrogen half-saturation constant (mgN/L). Use the ode45 and ode15s functions to solve these equations from t 5 0 to 50 d given the initial conditions a 5 1, n 5 4000, p 5 800, and c 5 0. Note that the parameters are kd 5 0.1, ks 5 0.15, kh 5 0.025, rnc 5 0.18, rpc 5 0.025, rna 5 7.2, rpa 5 1, rca 5 40, kg 5 0.5, ksp 5 2, and ksn 5 15. Develop plots of both solutions and interpret the results. 28.34 The following ODEs have been proposed as a model of an epidemic:

dS

dt 5 2aSI

dI

dt 5 aSI 2 rI

dR

dt 5 rI

where x 5 distance (m), K 5 hydraulic conductivity (m/d), h 5 height of the water table (m), h 5 the average height of the water table (m), and N 5 infi ltration rate (m/d).

Solve for the height of the water table for x 5 0 to 1000 m where h(0) 5 10 m and h(1000) 5 5 m. Use the following parameters for the calculation: K 5 1 m/d and N 5 0.0001 m/d. Set the average height of the water table as the average of the boundary conditions. Obtain your solution with (a) the shooting method and (b) the fi nite- difference method (Dx 5 100 m). 28.31 In Prob. 28.30, a linearized groundwater model was used to simulate the height of the water table for an unconfi ned aqui- fer. A more realistic result can be obtained by using the following nonlinear ODE:

d

dx aKh dh

dx b 1 N 5 0

where x 5 distance (m), K 5 hydraulic conductivity (m/d), h 5 height of the water table (m), and N 5 infi ltration rate (m/d). Solve for the height of the water table for the same case as in Prob. 28.30. That is solve from x 5 0 to 1000 m with h(0) 5 10 m, h(1000) 5 5 m, K 5 1 m/d, and N 5 0.0001 m/d. Obtain your solution with (a) the shooting method and (b) the fi nite-difference method (Dx 5 100 m). 28.32 The Lotka-Volterra equations described in Sec. 28.2 have been refi ned to include additional factors that impact predator-prey dynamics. For example, over and above predation, prey population can be limited by other factors such as space. Space limitation can be incorporated into the model as a carrying capacity (recall the logistic model described in Prob. 28.16) as in

dx

dt 5 a a1 2 x

K b x 2 bxy

Ground surface

Water table Infiltrationh

x

Confining bed

Aquifer Groundwater flow

FIGURE P28.30 An unconfi ned or “phreatic” aquifer.

cha9792x_ch28_811-840.indd Page 837 07/11/13 3:02 PM F-468 cha9792x_ch28_811-840.indd Page 837 07/11/13 3:02 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

838 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

D 5 2e dV

dx

where D is called the electric fl ux density vector, e 5 permittivity of the material, and V 5 electrostatic potential. Similarly, a Poisson equa- tion for electrostatic fi elds can be represented in one dimension as

d 2V

dx2 5 2

ry

e

where ry 5 charge density. Use the fi nite-difference technique with Dx 5 2 to determine V for a wire where V(0) 5 1000, V(20) 5 0, e 5 2, L 5 20, and ry 5 30.

Mechanical/Aerospace Engineering 28.41 Perform the same computation as in Sec. 28.4 but for a 1-m-long pendulum. 28.42 The rate of cooling of a body can be expressed as

dT

dt 5 2k(T 2 Ta)

where T 5 temperature of the body (8C), Ta 5 temperature of the sur- rounding medium (8C), and k 5 the proportionality constant (min21). Thus, this equation specifi es that the rate of cooling is proportional to the difference in temperature between the body and the surrounding medium. If a metal ball heated to 908C is dropped into water that is held at a constant value of Ta 5 208C, use a numerical method to compute how long it takes the ball to cool to 408C if k 5 0.25 min21. 28.43 The rate of heat fl ow (conduction) between two points on a cylinder heated at one end is given by

dQ

dt 5 l  A

dT

dx

where l 5 a constant, A 5 the cylinder’s cross-sectional area, Q 5 heat fl ow, T 5 temperature, t 5 time, and x 5 distance from the heated end. Because the equation involves two derivatives, we will simplify this equation by letting

dT

dx 5

100(L 2 x) (20 2 t)

100 2 xt

where L is the length of the rod. Combine the two equations and compute the heat fl ow for t 5 0 to 25 s. The initial condition is Q(0) 5 0 and the parameters are l 5 0.5 cal ? cm/s, A 5 12 cm2, L 5 20 cm, and x 5 2.5 cm. Plot your results. 28.44 Repeat the falling parachutist problem (Example 1.2), but with the upward force due to drag as a second-order rate:

Fu 5 2cy 2

where c 5 0.225 kg/m. Solve for t 5 0 to 30, plot your results, and compare with those of Example 1.2.

where S 5 the susceptible individuals, I 5 the infected, R 5 the recovered, a 5 the infection rate, and r 5 the recovery rate. A city has 10,000 people, all of whom are susceptible. (a) If a single infectious individual enters the city at t 5 0, com-

pute the progression of the epidemic until the number of in- fected individuals falls below 10. Use the following parameters: a 5 0.002/(person?week) and r 5 0.15yd. Develop time-series plots of all the state variables. Also generate a phaseplane plot of S versus I versus R.

(b) Suppose that after recovery, there is a loss of immunity that causes recovered individuals to become susceptible. This rein- fection mechanism can be computed as rR, where r 5 the reinfection rate. Modify the model to include this mechanism and repeat the computations in (a) using r 5 0.015yd.

Electrical Engineering 28.35 Perform the same computation as in the fi rst part of Sec. 28.3, but with R 5 0.025 V. 28.36 Solve the ODE in the fi rst part of Sec. 8.3 from t 5 0 to 0.5 using numerical techniques if q 5 0.1 and i 5 23.281515 at t 5 0. Use an R 5 50 along with the other parameters from Sec. 8.3. 28.37 For a simple RL circuit, Kirchhoff’s voltage law requires that (if Ohm’s law holds)

L di

dt 1 Ri 5 0

where i 5 current, L 5 inductance, and R 5 resistance. Solve for i, if L 5 1, R 5 1.5, and i(0) 5 0.5. Solve this problem analytically and with a numerical method. Present your results graphically. 28.38 In contrast to Prob. 28.37, real resistors may not always obey Ohm’s law. For example, the voltage drop may be nonlinear and the circuit dynamics is described by a relationship such as

L di

dt 1 R c i

I 2 a i

I b3 d 5 0

where all other parameters are as defi ned in Prob. 28.37 and I is a known reference current equal to 1. Solve for i as a function of time under the same conditions as specifi ed in Prob. 28.37. 28.39 Develop an eigenvalue problem for an LC network similar to the one in Fig. 28.14, but with only two loops. That is, omit the i3 loop. Draw the network, illustrating how the currents oscillate in their primary modes. 28.40 Just as Fourier’s law and the heat balance can be employed to characterize temperature distribution, analogous relationships are available to model fi eld problems in other areas of engineering. For example, electrical engineers use a similar approach when modeling electrostatic fi elds. Under a number of simplifying assumptions, an analog of Fourier’s law can be represented in one-dimensional form as

cha9792x_ch28_811-840.indd Page 838 06/11/13 8:22 PM F-468 cha9792x_ch28_811-840.indd Page 838 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 839

(d) The full nonlinear equation where both the damping and spring terms are nonlinear

m d 2x

dt2 1 a ` dx

dt ` dx dt

1 bx3 5 0

28.47 A forced damped spring-mass system (Fig. P28.47) has the following ordinary differential equation of motion:

m d 2x

dt2 1 a ` dx

dt ` dx dt

1 kx 5 Fo sin(vt)

where x 5 displacement from the equilibrium position, t 5 time, m 5 2 kg mass, a 5 5 N/(m/s)2, and k 5 6 N/m. The damping term is nonlinear and represents air damping. The forcing function Fo sin(vt) has values of Fo 5 2.5 N and v 5 0.5 rad/sec. The initial conditions are

Initial velocity dx

dt 5 0 m/s

Initial displacement x 5 1 m

Solve this equation using a numerical method over the time period 0 # t # 15 s. Plot the displacement and velocity versus time, and plot the forcing function on the same curve. Also, develop a sepa- rate plot of velocity versus displacement. 28.48 The temperature distribution in a tapered conical cooling fi n (Fig. P28.48) is described by the following differential equation, which has been nondimensionalized

d 2u

dx2 1 a2

x b adu

dx 2 pub 5 0

where u 5 temperature (0 # u # 1), x 5 axial distance (0 # x # 1), and p is a nondimensional parameter that describes the heat transfer and geometry

p 5 hL

k

B 1 1

4

2m2

28.45 Suppose that, after falling for 13 s, the parachutist from Examples 1.1 and 1.2 pulls the rip cord. At this point, assume that the drag coeffi cient is instantaneously increased to a constant value of 55 kg/s. Compute the parachutist’s velocity from t 5 0 to 30 s with Heun’s method (without iteration of the corrector) using a step-size of 2 s. Plot y versus t for t 5 0 to 30 s. 28.46 The following ordinary differential equation describes the motion of a damped spring-mass system (Fig. P28.46):

m d 2x

dt2 1 a ` dx

dt ` dx dt

1 bx3 5 0

where x 5 displacement from the equilibrium position, t 5 time, m 5 1 kg mass, and a 5 5 N/(m/s)2. The damping term is nonlinear and represents air damping.

The spring is a cubic spring and is also nonlinear with b 5 5 N/m3. The initial conditions are

Initial velocity dx

dt 5 0.5 mys

Initial displacement x 5 1 m

Solve this equation using a numerical method over the time period 0 # t # 8 s. Plot the displacement and velocity versus time and plot the phase-plane portrait (velocity versus displacement) for all the following cases: (a) A similar linear equation

m d 2x

dt2 1 2

dx

dt 1 5x 5 0

(b) The nonlinear equation with only a nonlinear spring term

d 2x

dt2 1 2

dx

dt 1 bx3 5 0

(c) The nonlinear equation with only a nonlinear damping term

m d 2x

dt2 1 a ` dx

dt ` dx dt

1 5x 5 0

Cubic spring

Air damping

x

m

FIGURE P28.46

Air damping

k

x

m Fo sin(�t)

FIGURE P28.47

cha9792x_ch28_811-840.indd Page 839 07/11/13 3:04 PM F-468 cha9792x_ch28_811-840.indd Page 839 07/11/13 3:04 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

840 CASE STUDIES: ORDINARY DIFFERENTIAL EQUATIONS

stretch, the spring and dampening forces of the cord must also be included. These two conditions can be expressed by the following equations:

dy

dt 5 g 2 sign(y)

cd m

y2 x # L

dy

dt 5 g 2 sign(y)

cd m

y2 2 k

m (x 2 L) 2

g

m y x . L

where y 5 velocity (m/s), t 5 time (s), g 5 gravitational constant (5 9.81 m/s2), sign(x) 5 function that returns 21, 0, and 1 for negative, zero, and positive x, respectively, cd 5 second-order drag coeffi cient (kg/m), m 5 mass (kg), k 5 cord spring constant (N/m), g 5 cord dampening coeffi cient (N ? s/m), and L 5 cord length (m). Determine the position and velocity of the jumper given the following parameters: L 5 30 m, m 5 68.1 kg, cd 5 0.25 kg/m, k 5 40 N/m, and g 5 8 kg/s. Perform the computation from t 5 0 to 50 s and assume that the initial conditions are x(0) 5 y(0) 5 0. 28.51 Two masses are attached to a wall by linear springs (Fig. P28.51). Force balances based on Newton’s second law can be written as

d 2x1 dt 2

5 2 k1 m1

(x1 2 L1) 1 k2 m1

(x2 2 x1 2 w1 2 L2)

d 2x2 dt 2

5 2 k2 m2

(x2 2 x1 2 w1 2 L2)

where k 5 the spring constants, m 5 mass, L 5 the length of the unstretched spring, and w 5 the width of the mass. Compute the positions of the masses as a function of time using the following pa- rameter values: k1 5 k2 5 5, m1 5 m2 5 2, w1 5 w2 5 5, and L1 5 L2 5 2. Set the initial conditions as x1 5 L1 and x2 5 L1 1 w1 1 L2 1 6. Perform the simulation from t 5 0 to 20. Construct time- series plots of both the displacements and the velocities. In addition, produce a phase-plane plot of x1 versus x2.

where h 5 a heat transfer coeffi cient, k 5 thermal conductivity, L 5 the length or height of the cone, and m 5 the slope of the cone wall. The equation has the boundary conditions

u(x 5 0) 5 0  u(x 5 1) 5 1

Solve this equation for the temperature distribution using fi nite difference methods. Use second-order accurate fi nite difference analogues for the derivatives. Write a computer program to obtain the solution and plot temperature versus axial distance for various values of p 5 10, 20, 50, and 100. 28.49 The dynamics of a forced spring-mass-damper system can be represented by the following second-order ODE:

m d 2x

dt2 1 c

dx

dt 1 k1x 1 k3x

3 5 P cos(vt)

where m 5 1 kg, c 5 0.4 N ? s/m, P 5 0.5 N, and v 5 0.5/s. Use a numerical method to solve for displacement (x) and velocity (y 5 dxydt) as a function of time with the initial conditions x 5 y 5 0. Express your results graphically as time-series plots (x and y versus t) and a phase plane plot (y versus x). Perform simulations for both (a) linear (k1 5 1; k3 5 0) and (b) nonlinear (k1 5 1; k3 5 0.5) springs. 28.50 The differential equation for the velocity of a bungee jumper is different depending on whether the jumper has fallen to a dis- tance where the cord is fully extended and begins to stretch. Thus, if the distance fallen is less than the cord length, the jumper is only subject to gravitational and drag forces. Once the cord begins to

x = 1

u(x = 1) = 1

u(x = 0) = 0

x

FIGURE P28.48

x

k1

L1 w1 L2 w2

x10 x2

k2 m1 m2

FIGURE P28.51

cha9792x_ch28_811-840.indd Page 840 06/11/13 8:22 PM F-468 cha9792x_ch28_811-840.indd Page 840 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

841

EPILOGUE: PART SEVEN

PT7.4 TRADE-OFFS

Table PT7.3 contains trade-offs associated with numerical methods for the solution of initial-value ordinary differential equations. The factors in this table must be evaluated by the engineer when selecting a method for each particular applied problem. Simple self-starting techniques such as Euler’s method can be used if the problem requirements involve a short range of integration. In this case, adequate accuracy may be obtained using small step sizes to avoid large truncation errors, and the round-off errors may be acceptable. Euler’s method may also be appropriate for cases where the mathematical model has an inherently high level of uncertainty or has coeffi cients or forcing functions with signifi cant errors as might arise during a measurement process. In this case, the accuracy of the model itself simply does not justify the effort involved to employ a more complicated numerical method. Finally, the simpler techniques may be best when the problem or simulation need only be performed a few times. In these applications, it is probably best to use a simple method that is easy to program and

TABLE PT7.3 Comparison of the characteristics of alternative methods for the numerical solution of ODEs. The comparisons are based on general experience and do not account for the behavior of special functions.

Starting Iterations Global Ease of Changing Programming Method Values Required Error Step Size Effort Comments

One step Euler’s 1 No O(h) Easy Easy Good for quick estimates Heun’s 1 Yes O(h2) Easy Moderate — Midpoint 1 No O(h2) Easy Moderate — Second-order Ralston 1 No O(h2) Easy Moderate The second-order RK

method that minimizes truncation error

Fourth-order RK 1 No O(h4) Easy Moderate Widely used Adaptive fourth-order RK or RK-Fehlberg 1 No O(h5)* Easy Moderate to Error estimate allows diffi cult step-size adjustment Multistep Non-self-starting 2 Yes O(h3)* Diffi cult Moderate to Simple multistep method Heun diffi cult†

Milne’s 4 Yes O(h5)* Diffi cult Moderate to Sometimes unstable diffi cult†

Fourth-order Adams 4 Yes O(h5)* Diffi cult Moderate to diffi cult†

*Provided the error estimate is used to modify the solution. †With variable step size.

cha9792x_ep07_841-843.indd Page 841 05/11/13 6:57 PM F-468 cha9792x_ep07_841-843.indd Page 841 05/11/13 6:57 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

842 EPILOGUE: PART SEVEN

understand despite the fact that the method may be computationally ineffi cient and rela- tively time-consuming to run on the computer. If the range of integration of the problem is long enough to involve a large number of steps, then it may be necessary and appropriate to use a more accurate technique than Euler’s method. The fourth-order RK method is popular and reliable for many engineer- ing problems. In these cases, it may also be advisable to estimate the truncation error for each step as a guide to selecting the best step size. This can be accomplished with the adaptive RK or fourth-order Adams approaches. If the truncation errors are extremely small, it may be wise to increase the step size to save computer time. On the other hand, if the truncation error is large, the step size should be decreased to avoid accumulation of error. Milne’s method should be avoided if signifi cant stability problems are expected. The Runge-Kutta method is simple to program and convenient to use but may be less effi cient than the multistep methods. However, the Runge-Kutta method is usually em- ployed in any event to obtain starting values for the multistep methods. A large number of engineering problems may fall into an intermediate range of in- tegration interval and accuracy requirement. In these cases, the second-order RK and the non-self-starting Heun methods are simple to use and are relatively effi cient and accurate. Stiff systems involve equations with slowly and rapidly varying components. Special techniques are usually required for the adequate solution of stiff equations. For example, implicit approaches are often used. You can consult Enright et al. (1975), Gear (1971), and Shampine and Gear (1979) for additional information regarding these techniques. A variety of techniques are available for solving eigenvalue problems. For small systems or where only a few of the smallest or largest eigenvalues are required, simple approaches such as the polynomial and the power methods are available. For symmetric systems, Jacobi’s, Given’s, or Householder’s method can be employed. Finally, the QR method represents a general approach for fi nding all the eigenvalues of symmetric and nonsymmetric matrices.

PT7.5 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT7.4 summarizes important information that was presented in Part Seven. This table can be consulted to quickly access important relationships and formulas.

PT7.6 ADVANCED METHODS AND ADDITIONAL REFERENCES

Although we have reviewed a number of techniques for solving ordinary differential equations there is additional information that is important in engineering practice. The question of stability was introduced in Sec. 26.2.4. This topic has general relevance to all methods for solving ODEs. Further discussion of the topic can be pursued in Carnahan, Luther, and Wilkes (1969), Gear (1971), and Hildebrand (1974). In Chap. 27, we introduced methods for solving boundary-value problems. Isaacson and Keller (1966), Keller (1968), Na (1979), and Scott and Watts (1976) can be consulted for additional information on standard boundary-value problems. Additional material on eigenvalues can be found in Ralston and Rabinowitz (1978), Wilkinson (1965), Fadeev and Fadeeva (1963), and Householder (1953, 1964). In summary, the foregoing is intended to provide you with avenues for deeper explo- ration of the subject. Additionally, all the above references provide descriptions of the basic techniques covered in Part Seven. We urge you to consult these alternative sources to broaden your understanding of numerical methods for the solution of differential equations.

cha9792x_ep07_841-843.indd Page 842 05/11/13 6:57 PM F-468 cha9792x_ep07_841-843.indd Page 842 05/11/13 6:57 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

8 4

3

TABLE PT7.4 Summary of important information presented in Part Seven.

Graphic Method Formulation Interpretation Errors

Euler (First- yi11 5 yi 1 hk1 Local error . O (h2) order RK) k1 5 f (xi, yi) Global error . O (h)

Ralston’s second- yi11 5 yi 1 h(13k1 1 2 3k2) Local error . O (h

3) order RK k1 5 f (xi, yi) Global error . O (h2) k2 5 f (xi 1 34h, yi 1

3 4hk1)

Classic fourth- yi11 5 yi 1 h(16k1 1 1 3k2 1

1 3k3 1

1 6k4) Local error . O (h

5) order RK k1 5 f (xi, yi) Global error . O (h4) k2 5 f (xi 1 12h, yi 1

1 2hk1)

k3 5 f (xi 1 12h, yi 1 1 2hk2)

k4 5 f (xi 1 h, yi 1 hk3)

Non-self-starting Predictor: (midpoint method) Predictor modifi er: Heun y 0i11 5 y mi21 1 2hf (xi, y mi ) Ep . 45(y

m i,u 1 y 0i,u )

Corrector: (trapezoidal rule) Corrector modifi er:

y ji11 5 y mi 1 h f (xi, y  mi ) 1 f (xi11, y i21i11)

2 Ec . 2

y mi11,u 2 y 0i11,u 5

Fourth-order Adams Predictor: (fourth Adams-Bashforth) Predictor modifi er: y 0i11 5 y mi 1 h(5524 f

m i 2

59 24 f

m i21 1

37 24 f

m i22 2

9 24 f

m i23) Ep . 251270 (y

m i,u 2 y 0i,u )

Corrector: (fourth Adams-Moulton) Corrector modifi er: y ji11 5 y mi 1 h( 924 f

j21 i11 1

19 24 f

m i 2

5 24 f

m i21 1

1 24 f

m i22) Ec . 2 19270 (y

m i11,u 2 y 0i11,u)

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

y

i – 3 i – 2 i – 1 i i + 1 x

cha9792x_ep07_841-843.indd Page 843 05/11/13 6:57 PM F-468

cha9792x_ep07_841-843.indd Page 843 05/11/13 6:57 PM F-468

/207/M H02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

/207/M H02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PART EIGHT

cha9792x_p08_844-851.indd Page 844 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 844 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

845

PT8.1 MOTIVATION

Given a function u that depends on both x and y, the partial derivative of u with respect to x at an arbitrary point (x, y) is defi ned as

0u 0x

5 lim ¢xS0

u(x 1 ¢x, y) 2 u(x, y) ¢x

(PT8.1)

Similarly, the partial derivative with respect to y is defi ned as

0u 0y

5 lim ¢yS0

u(x, y 1 ¢y) 2 u(x, y) ¢y

(PT8.2)

An equation involving partial derivatives of an unknown function of two or more inde- pendent variables is called a partial differential equation, or PDE. For example,

02u 0x2

1 2xy 02u 0y2

1 u 5 1 (PT8.3)

03u 0x20y

1 x 02u 0y2

1 8u 5 5y (PT8.4)

a 02u 0x2 b3 1 6 03u

0x0y2 5 x (PT8.5)

02u 0x2

1 xu 0u 0y

5 x (PT8.6)

The order of a PDE is that of the highest-order partial derivative appearing in the equa- tion. For example, Eqs. (PT8.3) and (PT8.4) are second- and third-order, respectively. A partial differential equation is said to be linear if it is linear in the unknown function and all its derivatives, with coeffi cients depending only on the independent variables. For example, Eqs. (PT8.3) and (PT8.4) are linear, whereas Eqs. (PT8.5) and (PT8.6) are not. Because of their widespread application in engineering, our treatment of PDEs will focus on linear, second-order equations. For two independent variables, such equations can be expressed in the following general form:

A 02u 0x2

1 B 02u

0x0y 1 C

02u 0y2

1 D 5 0 (PT8.7)

where A, B, and C are functions of x and y and D is a function of x, y, u, 0uy0x, and 0uy0y. Depending on the values of the coeffi cients of the second-derivative terms—A, B, C—

PARTIAL DIFFERENTIAL EQUATIONS

cha9792x_p08_844-851.indd Page 845 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 845 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

846 PARTIAL DIFFERENTIAL EQUATIONS

Eq. (PT8.7) can be classifi ed into one of three categories (Table PT8.1). This classifi cation, which is based on the method of characteristics (for example, see Vichnevetsky, 1981, or Lapidus and Pinder, 1981), is useful because each category relates to specifi c and distinct engineering problem contexts that demand special solution techniques. It should be noted that for cases where A, B, and C depend on x and y, the equation may actually fall into a different category, depending on the location in the domain for which the equation holds. For simplicity, we will limit the present discussion to PDEs that remain exclusively in one of the categories.

PT8.1.1 PDEs and Engineering Practice

Each of the categories of partial differential equations in Table PT8.1 conforms to specifi c kinds of engineering problems. The initial sections of the following chapters will be devoted to deriving each type of equation for a particular engineering problem context. For the time being, we will discuss their general properties and applications and show how they can be employed in different physical contexts. Elliptic equations are typically used to characterize steady-state systems. As in the Laplace equation in Table PT8.1, this is indicated by the absence of a time derivative. Thus, these equations are typically employed to determine the steady-state distribution of an unknown in two spatial dimensions. A simple example is the heated plate in Fig. PT8.1a. For this case, the boundaries of the plate are held at different temperatures. Because heat fl ows from regions of high to low temperature, the boundary conditions set up a potential that leads to heat fl ow from the hot to the cool boundaries. If suffi cient time elapses, such a system will even- tually reach the stable or steady-state distribution of temperature depicted in Fig. PT8.1a. The Laplace equation, along with appropriate boundary conditions, provides a means to determine this distribution. By analogy, the same approach can be employed to tackle other problems involving potentials, such as seepage of water under a dam (Fig. PT8.1b) or the distribution of an electric fi eld (Fig. PT8.1c).

TABLE PT8.1 Categories into which linear, second-order partial differential equations in two variables can be classifi ed.

B2 2 4AC Category Example

, 0 Elliptic Laplace equation (steady state with two spatial dimensions)

02T 0x2

1 02T 0y 2

5 0

5 0 Parabolic Heat conduction equation (time variable with one spatial dimension)

0T 0t

5 k¿ 02T 0x2

. 0 Hyperbolic Wave equation (time variable with one spatial dimension)

02y

0x2 5

1 c2

02y

0t  2

cha9792x_p08_844-851.indd Page 846 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 846 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT8.1 MOTIVATION 847

In contrast to the elliptic category, parabolic equations determine how an unknown varies in both space and time. This is manifested by the presence of both spatial and tem- poral derivatives in the heat conduction equation from Table PT8.1. Such cases are referred to as propagation problems because the solution “propagates,’’ or changes, in time. A simple example is a long, thin rod that is insulated everywhere except at its end (Fig. PT8.2a). The insulation is employed to avoid complications due to heat loss along

Con duc

tor

Dam

Flow line

Impermeable rock

Equipotential line

Hot

Cool

CoolHot

(a) (b) (c)

FIGURE PT8.1 Three steady-state distribution problems that can be characterized by elliptic PDEs. (a) Temperature distribution on a heated plate, (b) seepage of water under a dam, and (c) the electric fi eld near the point of a conductor.

FIGURE PT8.2 (a) A long, thin rod that is insulated everywhere but at its end. The dynamics of the one- dimensional distribution of temperature along the rod’s length can be described by a parabolic PDE. (b) The solution, consisting of distributions corresponding to the state of the rod at various times.

T

x

(a)

(b)

CoolHot

t = 3�t

t = 2�tt = �tt = 0

cha9792x_p08_844-851.indd Page 847 18/12/13 9:41 AM F-468 cha9792x_p08_844-851.indd Page 847 18/12/13 9:41 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

848 PARTIAL DIFFERENTIAL EQUATIONS

the rod’s length. As was the case for the heated plate in Fig. PT8.1a, the ends of the rod are set at fi xed temperatures. However, in contrast to Fig. PT8.1a, the rod’s thinness allows us to assume that heat is distributed evenly over its cross section—that is, later- ally. Consequently, lateral heat fl ow is not an issue, and the problem reduces to studying the conduction of heat along the rod’s longitudinal axis. Rather than focusing on the steady-state distribution in two spatial dimensions, the problem shifts to determining how the one-dimensional spatial distribution changes as a function of time (Fig. PT8.2b). Thus, the solution consists of a series of spatial distributions corresponding to the state of the rod at various times. Using an analogy from photography, the elliptic case yields a portrait of a system’s stable state, whereas the parabolic case provides a motion picture of how it changes from one state to another. As with the other types of PDEs described herein, parabolic equations can be used to characterize a wide variety of other engineer- ing problem contexts by analogy. The fi nal class of PDEs, the hyperbolic category, also deals with propagation prob- lems. However, an important distinction manifested by the wave equation in Table PT8.1 is that the unknown is characterized by a second derivative with respect to time. As a consequence, the solution oscillates. The vibrating string in Fig. PT8.3 is a simple physical model that can be described with the wave equation. The solution consists of a number of characteristic states with which the string oscillates. A variety of engineering systems such as vibrations of rods and beams, motion of fl uid waves, and transmission of sound and electrical signals can be characterized by this model.

PT8.1.2 Precomputer Methods for Solving PDEs

Prior to the advent of digital computers, engineers relied on analytical or exact solutions of partial differential equations. Aside from the simplest cases, these solutions often required a great deal of effort and mathematical sophistication. In addition, many phys- ical systems could not be solved directly but had to be simplifi ed using linearizations, simple geometric representations, and other idealizations. Although these solutions are elegant and yield insight, they are limited with respect to how faithfully they represent real systems—especially those that are highly nonlinear and irregularly shaped.

PT8.2 ORIENTATION

Before we proceed to the numerical methods for solving partial differential equations, some orientation might be helpful. The following material is intended to provide you with an overview of the material discussed in Part Eight. In addition, we have formulated objectives to focus your studies in the subject area.

FIGURE PT8.3 A taut string vibrating at a low amplitude is a simple physical system that can be characterized by a hyperbolic PDE.

cha9792x_p08_844-851.indd Page 848 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 848 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT8.2 ORIENTATION 849

PT8.2.1 Scope and Preview

Figure PT8.4 provides an overview of Part Eight. Two broad categories of numerical methods will be discussed in this part of this book. Finite-difference approaches, which are covered in Chaps. 29 and 30, are based on approximating the solution at a fi nite number of points. In contrast, fi nite-element methods, which are covered in Chap. 31,

CHAPTER 29 Finite Difference:

Elliptic Equations

PART EIGHT Partial

Differential Equations

CHAPTER 30 Finite Difference:

Parabolic Equations

CHAPTER 31 Finite-Element

Method

CHAPTER 32 Case Studies

EPILOGUE

30.4 Crank-

Nicolson

30.3 Simple implicit

methods

30.2 Explicit

methods

30.1 Heat-conduction

equation

PT 8.5 Advanced methods

PT 8.4 Important formulas

32.4 Mechanical engineering

32.3 Electrical

engineering

32.2 Civil

engineering 32.1

Chemical engineering

31.4 Software packages

31.1 General

approach

31.3 Two-dimensional

analysis

31.2 One-dimensional

analysis

PT 8.3 Trade-offs

PT 8.2 Orientation

PT 8.1 Motivation

29.2 Finite-difference

solution

29.3 Boundary conditions

29.4 Control-volume

approach

29.5 Computer algorithms

29.1 Laplace equation

30.5 ADI

FIGURE PT8.4 Schematic representation of the organization of material in Part Eight: Partial Differential Equations.

cha9792x_p08_844-851.indd Page 849 18/12/13 9:41 AM F-468 cha9792x_p08_844-851.indd Page 849 18/12/13 9:41 AM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

850 PARTIAL DIFFERENTIAL EQUATIONS

approximate the solution in pieces, or “elements.” Various parameters are adjusted until these approximations conform to the underlying differential equation in an optimal sense. Chapter 29 is devoted to fi nite-difference solutions of elliptic equations. Before launching into the methods, we derive the Laplace equation for the physical problem context of the temperature distribution for a heated plate. Then, a standard solution ap- proach, the Liebmann method, is described. We will illustrate how this approach is used to compute the distribution of the primary scalar variable, temperature, as well as a secondary vector variable, heat fl ux. The fi nal section of the chapter deals with boundary conditions. This material includes procedures to handle different types of conditions as well as irregular boundaries. In Chap. 30, we turn to fi nite-difference solutions of parabolic equations. As with the discussion of elliptic equations, we fi rst provide an introduction to a physical problem context, the heat-conduction equation for a one-dimensional rod. Then we introduce both explicit and implicit algorithms for solving this equation. This is followed by an effi cient and reliable implicit method—the Crank-Nicolson technique. Finally, we describe a particu- larly effective approach for solving two-dimensional parabolic equations—the alternating- direction implicit, or ADI, method. Note that, because they are somewhat beyond the scope of this book, we have chosen to omit hyperbolic equations. The epilogue of this part of the book contains references related to this type of PDE. In Chap. 31, we turn to the other major approach for solving PDEs—the fi nite-element method. Because it is so fundamentally different from the fi nite-difference approach, we devote the initial section of the chapter to a general overview. Then we show how the fi nite- element method is used to compute the steady-state temperature distribution of a heated rod. Finally, we provide an introduction to some of the issues involved in extending such an analysis to two-dimensional problem contexts. Chapter 32 is devoted to applications from all fi elds of engineering. Finally, a short review section is included at the end of Part Eight. This epilogue summarizes important information related to PDEs. This material includes a discussion of trade-offs that are rel- evant to their implementation in engineering practice. The epilogue also includes references for advanced topics.

PT8.2.2 Goals and Objectives

Study Objectives. After completing Part Eight, you should have greatly enhanced your capability to confront and solve partial differential equations. General study goals should include mastering the techniques, having the capability to assess the reliability of the an- swers, and being able to choose the “best’’ method (or methods) for any particular problem. In addition to these general objectives, the specifi c study objectives in Table PT8.2 should be mastered.

Computer Objectives. Computer algorithms can be developed for many of the methods in Part Eight. For example, you may fi nd it instructive to develop a general program to simulate the steady-state distribution of temperature on a heated plate. Further, you might want to develop programs to implement both the simple explicit and the Crank-Nicolson methods for solving parabolic PDEs in one spatial dimension.

cha9792x_p08_844-851.indd Page 850 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 850 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PT8.2 ORIENTATION 851

Finally, one of your most important goals should be to master several of the general- purpose software packages that are widely available. In particular, you should become adept at using these tools to implement numerical methods for engineering problem solving.

TABLE PT8.2 Specifi c study objectives for Part Eight.

1. Recognize the difference between elliptic, parabolic, and hyperbolic PDEs. 2. Understand the fundamental difference between fi nite-difference and fi nite-element approaches. 3. Recognize that the Liebmann method is equivalent to the Gauss-Seidel approach for solving

simultaneous linear algebraic equations. 4. Know how to determine secondary variables for two-dimensional fi eld problems. 5. Recognize the distinction between Dirichlet and derivative boundary conditions. 6. Understand how to use weighting factors to incorporate irregular boundaries into a fi nite-difference

scheme for PDEs. 7. Understand how to implement the control-volume approach for implementing numerical solutions

of PDEs. 8. Know the difference between convergence and stability of parabolic PDEs. 9. Understand the difference between explicit and implicit schemes for solving parabolic PDEs. 10. Recognize how the stability criteria for explicit methods detract from their utility for solving

parabolic PDEs. 11. Know how to interpret computational molecules. 12. Recognize how the ADI approach achieves high effi ciency in solving parabolic equations in two

spatial dimensions. 13. Understand the difference between the direct method and the method of weighted residuals for

deriving element equations. 14. Know how to implement Galerkin’s method. 15. Understand the benefi ts of integration by parts during the derivation of element equations; in

particular, recognize the implications of lowering the highest derivative from a second to a fi rst derivative.

cha9792x_p08_844-851.indd Page 851 05/11/13 6:56 PM F-468 cha9792x_p08_844-851.indd Page 851 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29

852

C H A P T E R 29

Finite Difference: Elliptic Equations

Elliptic equations in engineering are typically used to characterize steady-state, boundary- value problems. Before demonstrating how they can be solved, we will illustrate how a simple case—the Laplace equation—is derived from a physical problem context.

29.1 THE LAPLACE EQUATION

As mentioned in the introduction to this part of the book, the Laplace equation can be used to model a variety of problems involving the potential of an unknown variable. Because of its simplicity and general relevance to most areas of engineering, we will use a heated plate as our fundamental context for deriving and solving this elliptic PDE. Homework problems and engineering applications (Chap. 32) will be employed to illus- trate the applicability of the model to other engineering problem contexts. Figure 29.1 shows an element on the face of a thin rectangular plate of thickness Dz. The plate is insulated everywhere but at its edges, where the temperature can be set at a prescribed level. The insulation and the thinness of the plate mean that heat transfer is limited to the x and y dimensions. At steady state, the fl ow of heat into the element over a unit time period Dt must equal the fl ow out, as in

q(x)¢y ¢z ¢t 1 q(y) ¢x ¢z ¢t 5 q(x 1 ¢x)¢y ¢z ¢t 1 q(y 1 ¢y)¢x ¢z ¢t (29.1)

where q(x) and q(y) 5 the heat fl uxes at x and y, respectively [cal/(cm2 ? s)]. Dividing by Dz and Dt and collecting terms yields

[q(x) 2 q(x 1 ¢x)]¢y 1 [q(y) 2 q(y 1 ¢y)]¢x 5 0

Multiplying the fi rst term by DxyDx and the second by DyyDy gives

q(x) 2 q(x 1 ¢x) ¢x

¢x ¢y 1 q(y) 2 q(y 1 ¢y)

¢y ¢y ¢x 5 0 (29.2)

Dividing by Dx Dy and taking the limit results in

2 0q 0x

2 0q 0y

5 0 (29.3)

where the partial derivatives result from the defi nitions in Eqs. (PT8.1) and (PT8.2).

cha9792x_ch29_852-872.indd Page 852 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 852 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.1 THE LAPLACE EQUATION 853

Equation (29.3) is a partial differential equation that is an expression of the conserva- tion of energy for the plate. However, unless heat fl uxes are specifi ed at the plate’s edges, it cannot be solved. Because temperature boundary conditions are given, Eq. (29.3) must be reformulated in terms of temperature. The link between fl ux and temperature is pro- vided by Fourier’s law of heat conduction, which can be represented as

qi 5 2krC 0T 0i

(29.4)

where qi 5 heat fl ux in the direction of the i dimension [cal/(cm 2 ? s)], k 5 coeffi cient

of thermal diffusivity (cm2/s), r 5 density of the material (g/cm3), C 5 heat capacity of the material [cal/(g ? 8C)], and T 5 temperature (8C), which is defi ned as

T 5 H

rCV

where H 5 heat (cal) and V 5 volume (cm3). Sometimes the term in front of the dif- ferential in Eq. (29.4) is treated as a single term,

k¿ 5 krC (29.5)

where k9 is referred to as the coeffi cient of thermal conductivity [cal/(s ? cm ? 8C)]. In either case, both k and k9 are parameters that refl ect how well the material conducts heat. Fourier’s law is sometimes referred to as a constitutive equation. It is given this label because it provides a mechanism that defi nes the system’s internal interactions. Inspec- tion of Eq. (29.4) indicates that Fourier’s law specifi es that heat fl ux perpendicular to

FIGURE 29.1 A thin plate of thickness Dz. An element is shown about which a heat balance is taken.

�z

�x

�y

q(y)

q(x) q(x + �x)

q(y + �y)

y

x

cha9792x_ch29_852-872.indd Page 853 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 853 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

854 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

the i axis is proportional to the gradient or slope of temperature in the i direction. The negative sign ensures that a positive fl ux in the direction of i results from a negative slope from high to low temperature (Fig. 29.2). Substituting Eq. (29.4) into Eq. (29.3) results in

02T 0x2

1 02T 0y2

5 0 (29.6)

which is the Laplace equation. Note that for the case where there are sources or sinks of heat within the two-dimensional domain, the equation can be represented as

02T 0x2

1 02T 0y2

5 f(x, y) (29.7)

where f(x, y) is a function describing the sources or sinks of heat. Equation (29.7) is referred to as the Poisson equation.

29.2 SOLUTION TECHNIQUE

The numerical solution of elliptic PDEs such as the Laplace equation proceeds in the reverse manner of the derivation of Eq. (29.6) from the preceding section. Recall that the derivation of Eq. (29.6) employed a balance around a discrete element to yield an algebraic difference equation characterizing heat fl ux for a plate. Taking the limit turned this difference equation into a differential equation [Eq. (29.3)]. For the numerical solution, fi nite-difference representations based on treating the plate as a grid of discrete points (Fig. 29.3) are substituted for the partial derivatives in Eq. (29.6). As described next, the PDE is transformed into an algebraic difference equation.

FIGURE 29.2 Graphical depiction of a temperature gradient. Because heat moves ”downhill” from high to low temperature, the fl ow in (a) is from left to right in the positive i direction. However, due to the ori- entation of Cartesian coordinates, the slope is negative for this case. Thus, a negative gradient leads to a positive fl ow. This is the origin of the minus sign in Fourier’s law of heat conduction. The reverse case is depicted in (b), where the positive gradient leads to a negative heat fl ow from right to left.

T

i

(b)(a)

Direction of heat flow

T

i

Direction of heat flow

�T �i

� 0

�T �i

� 0

cha9792x_ch29_852-872.indd Page 854 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 854 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.2 SOLUTION TECHNIQUE 855

29.2.1 The Laplacian Difference Equation

Central differences based on the grid scheme from Fig. 29.3 are (recall Fig. 23.3)

02T 0x2

5 Ti11, j 2 2Ti, j 1 Ti21, j

¢x2

and

02T 0y2

5 Ti, j11 2 2Ti, j 1 Ti, j21

¢y2

which have errors of O[D(x)2] and O[D(y)2], respectively. Substituting these expressions into Eq. (29.6) gives

Ti11, j 2 2Ti, j 1 Ti21, j

¢x2 1

Ti, j11 2 2Ti, j 1 Ti, j21

¢y2 5 0

For the square grid in Fig. 29.3, Dx 5 Dy, and by collection of terms, the equation becomes

Ti11, j 1 Ti21, j 1 Ti, j11 1 Ti, j21 2 4Ti, j 5 0 (29.8)

This relationship, which holds for all interior points on the plate, is referred to as the Laplacian difference equation. In addition, boundary conditions along the edges of the plate must be specifi ed to obtain a unique solution. The simplest case is where the temperature at the boundary is set at a fi xed value. This is called a Dirichlet boundary condition. Such is the case for Fig. 29.4, where

FIGURE 29.3 A grid used for the fi nite-difference solution of elliptic PDEs in two independent variables such as the Laplace equation.

y

x

i – 1, j i + 1, j

0, n + 1 m + 1, n + 1

m + 1, 0

i, j – 1

i, j + 1

0, 0

i, j

cha9792x_ch29_852-872.indd Page 855 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 855 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

856 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

the edges are held at constant temperatures. For the case illustrated in Fig. 29.4, a balance for node (1, 1) is, according to Eq. (29.8),

T21 1 T01 1 T12 1 T10 2 4T11 5 0 (29.9)

However, T01 5 75 and T10 5 0, and therefore, Eq. (29.9) can be expressed as

24T11 1 T12 1 T21 5 275

Similar equations can be developed for the other interior points. The result is the following set of nine simultaneous equations with nine unknowns:

4T11 2T21 2T12 5 75

2T11 14T21 2T31 2T22 5 0

2T21 14T31 2T32 5 50

2T11 14T12 2T22 2T13 5 75

2T21 2T12 14T22 2T32 2T23 5 0

2T31 2T22 14T32 2T33 5 50

2T12 14T13 2T23 5 175

2T22 2T13 14T23 2T33 5 100

2T32 2T23 14T33 5 150

(29.10)

29.2.2 The Liebmann Method

Most numerical solutions of the Laplace equation involve systems that are much larger than Eq. (29.10). For example, a 10-by-10 grid involves 100 linear algebraic equations. Solution techniques for these types of equations were discussed in Part Three.

FIGURE 29.4 A heated plate where boundary temperatures are held at constant levels. This case is called a Dirichlet boundary condition.

(1, 3) (2, 3) (3, 3)

(1, 2) (2, 2) (3, 2)

(1, 1) (2, 1) (3, 1)

100�C

0�C

75�C 50�C

cha9792x_ch29_852-872.indd Page 856 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 856 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.2 SOLUTION TECHNIQUE 857

Notice that there are a maximum of fi ve unknown terms per line in Eq. (29.10). For larger-sized grids, this means that a signifi cant number of the terms will be zero. When applied to such sparse systems, full-matrix elimination methods waste great amounts of computer memory storing these zeros. For this reason, approximate methods provide a viable approach for obtaining solutions for elliptical equations. The most commonly employed approach is Gauss-Seidel, which when applied to PDEs is also referred to as Liebmann’s method. In this technique, Eq. (29.8) is expressed as

Ti, j 5 Ti11, j 1 Ti21, j 1 Ti, j11 1 Ti, j21

4 (29.11)

and solved iteratively for j 5 1 to n and i 5 1 to m. Because Eq. (29.8) is diagonally dominant, this procedure will eventually converge on a stable solution (recall Sec. 11.2.1). Overrelaxation is sometimes employed to accelerate the rate of convergence by applying the following formula after each iteration:

T newi, j 5 lT new i, j 1 (1 2 l)T

old i, j (29.12)

where T newi, j and T old i, j are the values of Ti, j from the present and the previous iteration,

respectively, and l is a weighting factor that is set between 1 and 2. As with the conventional Gauss-Seidel method, the iterations are repeated until the absolute values of all the percent relative errors (ea)i, j fall below a prespecifi ed stopping criterion es. These percent relative errors are estimated by

Z (ea)i, j Z 5 ` T new i, j 2 T

old i, j

T newi, j ` 100% (29.13)

EXAMPLE 29.1 Temperature of a Heated Plate with Fixed Boundary Conditions

Problem Statement. Use Liebmann’s method (Gauss-Seidel) to solve for the tempera- ture of the heated plate in Fig. 29.4. Employ overrelaxation with a value of 1.5 for the weighting factor and iterate to es 5 1%.

Solution. Equation (29.11) at i 5 1, j 5 1 is

T11 5 0 1 75 1 0 1 0

4 5 18.75

and applying overrelaxation yields

T11 5 1.5(18.75) 1 (1 2 1.5)0 5 28.125

For i 5 2, j 5 1,

T21 5 0 1 28.125 1 0 1 0

4 5 7.03125

T21 5 1.5(7.03125) 1 (1 2 1.5)0 5 10.54688

For i 5 3, j 5 1,

T31 5 50 1 10.54688 1 0 1 0

4 5 15.13672

cha9792x_ch29_852-872.indd Page 857 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 857 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

858 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

T31 5 1.5(15.13672) 1 (1 2 1.5)0 5 22.70508

The computation is repeated for the other rows to give

T12 5 38.67188  T22 5 18.45703  T32 5 34.18579

T13 5 80.12696  T23 5 74.46900  T33 5 96.99554

Because all the Ti, j’s are initially zero, all ea’s for the fi rst iteration will be 100%. For the second iteration the results are

T11 5 32.51953  T21 5 22.35718  T31 5 28.60108

T12 5 57.95288  T22 5 61.63333  T32 5 71.86833

T13 5 75.21973  T23 5 87.95872  T32 5 67.68736

The error for T1,1 can be estimated as [Eq. (29.13)]

Z (ea)1, 1 Z 5 ` 32.51953 2 28.1250032.51953 ` 100% 5 13.5% Because this value is above the stopping criterion of 1%, the computation is continued. The ninth iteration gives the result

T11 5 43.00061  T21 5 33.29755  T31 5 33.88506

T12 5 63.21152  T22 5 56.11238  T32 5 52.33999

T13 5 78.58718  T23 5 76.06402  T33 5 69.71050

where the maximum error is 0.71%. Figure 29.5 shows the results. As expected, a gradient is established as heat fl ows from high to low temperatures.

FIGURE 29.5 Temperature distribution for a heated plate subject to fi xed boundary conditions.

78.59 76.06 69.71

63.21 56.11 52.34

43.00 33.30 33.89

100�C

0�C

75�C 50�C

cha9792x_ch29_852-872.indd Page 858 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 858 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.2 SOLUTION TECHNIQUE 859

29.2.3 Secondary Variables

Because its distribution is described by the Laplace equation, temperature is considered to be the primary variable in the heated-plate problem. For this case, as well as for other problems involving PDEs, secondary variables may also be of interest. As a matter of fact, in certain engineering contexts, the secondary variable may actually be more important. For the heated plate, a secondary variable is the rate of heat fl ux across the plate’s surface. This quantity can be computed from Fourier’s law. Central fi nite-difference ap- proximations for the fi rst derivatives (recall Fig. 23.3) can be substituted into Eq. (29.4) to give the following values for heat fl ux in the x and y dimensions:

qx 5 2k¿ Ti11, j 2 Ti21, j

2 ¢x (29.14)

and

qy 5 2k¿ Ti, j11 2 Ti, j21

2 ¢y (29.15)

The resultant heat fl ux can be computed from these two quantities by

qn 5 2q2x 1 q2y (29.16)

where the direction of qn is given by

u 5 tan21 aqy qx b (29.17)

for qx . 0 and

u 5 tan21 aqy qx b 1 p (29.18)

for qx , 0. Recall that the angle can be expressed in degrees by multiplying it by 1808/p. If qx 5 0, u is py2 (908) or 3py2 (2708), depending on whether qy is positive or negative, respectively.

EXAMPLE 29.2 Flux Distribution for a Heated Plate

Problem Statement. Employ the results of Example 29.1 to determine the distribution of heat fl ux for the heated plate from Fig. 29.4. Assume that the plate is 40 3 40 cm and is made out of aluminum [k9 5 0.49 cal/(s ? cm ? 8C)].

Solution. For i 5 j 5 1, Eq. (29.14) can be used to compute

qx 5 20.49 cal

s # cm # °C (33.29755 2 75)°C

2(10 cm) 5 1.022 cal/ (cm2 # s)

and [Eq. (29.15)]

qy 5 20.49 cal

s # cm # °C (63.21152 2 0)°C

2(10 cm) 5 21.549 cal/ (cm2 # s)

cha9792x_ch29_852-872.indd Page 859 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 859 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

860 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

The resultant fl ux can be computed with Eq. (29.16):

qn 5 2(1.022) 2 1 (21.549)2 5 1.856 cal/ (cm2 # s)

and the angle of its trajectory by Eq. (29.17)

u 5 tan21 a21.549 1.022

b 5 20.98758 3 180° p

5 256.584°

Thus, at this point, the heat fl ux is directed down and to the right. Values at the other grid points can be computed; the results are displayed in Fig. 29.6.

29.3 BOUNDARY CONDITIONS

Because it is free of complicating factors, the rectangular plate with fi xed boundary condi- tions has been an ideal context for showing how elliptic PDEs can be solved numerically. We will now elaborate on other issues that will expand our capabilities to address more realistic problems. These involve boundaries at which the derivative is specifi ed and bound- aries that are irregularly shaped.

29.3.1 Derivative Boundary Conditions

The fi xed or Dirichlet boundary condition discussed to this point is but one of several types that are used with partial differential equations. A common alternative is the case where

FIGURE 29.6 Heat fl ux for a plate subject to fi xed boundary temperatures. Note that the lengths of the arrows are proportional to the magnitude of the fl ux.

100�C

0�C

75�C 50�C

cha9792x_ch29_852-872.indd Page 860 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 860 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.3 BOUNDARY CONDITIONS 861

the derivative is given. This is commonly referred to as a Neumann boundary condition. For the heated-plate problem, this amounts to specifying the heat fl ux rather than the tem- perature at the boundary. One example is the situation where the edge is insulated. In this case, the derivative is zero. This conclusion is drawn directly from Eq. (29.4) because insulating a boundary means that the heat fl ux (and consequently the gradient) must be zero. Another example would be where heat is lost across the edge by predictable mecha- nisms such as radiation or convection. Figure 29.7 depicts a node (0, j) at the left edge of a heated plate. Applying Eq. (29.8) at the point gives

T1, j 1 T21, j 1 T0, j11 1 T0, j21 2 4T0, j 5 0 (29.19)

Notice that an imaginary point (21, j) lying outside the plate is required for this equa- tion. Although this exterior fi ctitious point might seem to represent a problem, it actually serves as the vehicle for incorporating the derivative boundary condition into the prob- lem. This is done by representing the fi rst derivative in the x dimension at (0, j) by the fi nite divided difference

0T 0x

> T1, j 2 T21, j

2 ¢x

which can be solved for

T21, j 5 T1, j 2 2 ¢x 0T 0x

Now we have a relationship for T2l, j that actually includes the derivative. It can be sub- stituted into Eq. (29.19) to give

2T1, j 2 2 ¢x 0T 0x

1 T0, j11 1 T0, j21 2 4T0, j 5 0 (29.20)

Thus, we have incorporated the derivative into the balance. Similar relationships can be developed for derivative boundary conditions at the other edges. The following example shows how this is done for the heated plate.

EXAMPLE 29.3 Heated Plate with an Insulated Edge

Problem Statement. Repeat the same problem as in Example 29.1, but with the lower edge insulated.

Solution. The general equation to characterize a derivative at the lower edge (that is, at j 5 0) of a heated plate is

Ti11, 0 1 Ti21, 0 1 2Ti, 1 2 2 ¢y 0T 0y

2 4Ti, 0 5 0

For an insulated edge, the derivative is zero and the equation becomes

Ti11, 0 1 Ti21, 0 1 2Ti, 1 2 4Ti, 0 5 0

T0, j + 1

T0, j – 1

T–1, j T0, j T1, j

FIGURE 29.7 A boundary node (0, j) on the left edge of a heated plate. To approximate the derivative nor- mal to the edge (that is, the x derivative), an imaginary point (21, j) is located a distance Dx beyond the edge.

cha9792x_ch29_852-872.indd Page 861 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 861 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

862 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

The simultaneous equations for temperature distribution on the plate in Fig. 29.4 with an insulated lower edge can be written in matrix form as

4 21 22

21 4 21 22

21 4 22

21 4 21 21

21 21 4 21 21

21 21 4 21 21

21 4 21 21

21 21 4 21 21

21 21 4 21

21 4 21

21 21 4 21

21 21 4

 

T10 T20 T30 T11 T21 T31 T12 T22 T32 T13 T23 T33

5

75

0

50

75

0

50

75

0

50

175

100

150

Note that because of the derivative boundary condition, the matrix is increased to 12 3 12 in contrast to the 9 3 9 system in Eq. (29.10) to account for the three unknown temperatures along the plate’s lower edge. These equations can be solved for

T10 5 71.91  T20 5 67.01  T30 5 59.54

T11 5 72.81  T21 5 68.31  T31 5 60.57

T12 5 76.01  T22 5 72.84  T32 5 64.42

T13 5 83.41  T23 5 82.63  T33 5 74.26

75

75

75

75

50

50

50

50

100 100 100

83.4 82.6 74.3

76.0 72.8 64.4

72.8 68.3 60.6

71.9 67.0 59.5

Insulated

FIGURE 29.8 Temperature and fl ux distribution for a heated plate subject to fi xed boundary conditions except for an insulated lower edge.

cha9792x_ch29_852-872.indd Page 862 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 862 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.3 BOUNDARY CONDITIONS 863

These results and computed fl uxes (for the same parameters as in Example 29.2) are displayed in Fig. 29.8. Note that, because the lower edge is insulated, the plate’s tem- perature is higher than for Fig. 29.5, where the bottom edge temperature is fi xed at zero. In addition, the heat fl ow (in contrast to Fig. 29.6) is now defl ected to the right and moves parallel to the insulated wall.

29.3.2 Irregular Boundaries

Although the rectangular plate from Fig. 29.4 has served well to illustrate the fundamental aspects of solving elliptic PDEs, many engineering problems do not exhibit such an ideal- ized geometry. For example, a great many systems have irregular boundaries (Fig. 29.9). Figure 29.9 is a system that can serve to illustrate how nonrectangular boundaries can be handled. As depicted, the plate’s lower left boundary is circular. Notice that we have affi xed parameters—a1, a2, b1, b2—to each of the lengths surrounding the node. Of course, for the plate depicted in Fig. 29.9, a2 5 b2 5 1. However, we will retain these parameters throughout the following derivation so that the resulting equation is generally applicable to any irregular boundary—not just one on the lower left-hand corner of a heated plate. The fi rst derivatives in the x dimension can be approximated as

a 0T 0x b

i21, i >

Ti, j 2 Ti21, j

a1 ¢x (29.21)

and

a 0T 0x b

i, i11 >

Ti11, j 2 Ti, j

a2 ¢x (29.22)

FIGURE 29.9 A grid for a heated plate with an irregularly shaped boundary. Note how weighting coeffi cients are used to account for the nonuniform spacing in the vicinity of the nonrectangular boundary.

�2 �y

�1 �y

�1 �x �2 �x

cha9792x_ch29_852-872.indd Page 863 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 863 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

864 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

The second derivatives can be developed from these fi rst derivatives. For the x dimension, the second derivative is

02T 0x2

5 0 0x

a 0T 0x b 5 a 0T 0x b

i, i11 2 a 0T 0x b

i21, i

a1 ¢x 1 a2 ¢x 2

(29.23)

Substituting Eqs. (29.21) and (29.22) into (29.23) gives

02T 0x2

5 2

Ti21, j 2 Ti, j

a1 ¢x 2

Ti11, j 2 Ti, j

a2 ¢x a1 ¢x 1 a2 ¢x

Collecting terms yields

02T 0x2

5 2

¢x2 c Ti21, j 2 Ti, j a1(a1 1 a2)

1 Ti11, j 2 Ti, j

a2(a1 1 a2) d

A similar equation can be developed in the y dimension:

02T 0y2

5 2

¢y2 c Ti, j21 2 Ti, j b1(b1 1 b2)

1 Ti, j11 2 Ti, j

b2(b1 1 b2) d

Substituting these equations in Eq. (29.6) yields

2

¢x2 c Ti21, j 2 Ti, j a1(a1 1 a2)

1 Ti11, j 2 Ti, j

a2(a1 1 a2) d

1 2

¢y2 c Ti, j21 2 Ti, j b1(b1 1 b2)

1 Ti, j11 2 Ti, j

b2(b1 1 b2) d 5 0 (29.24)

As illustrated in the following example, Eq. (29.24) can be applied to any node that lies adjacent to an irregular, Dirichlet-type boundary.

EXAMPLE 29.4 Heated Plate with an Irregular Boundary

Problem Statement. Repeat the same problem as in Example 29.1, but with the lower edge as depicted in Fig. 29.9.

Solution. For the case in Fig. 29.9, Dx 5 Dy, a1 5 b1 5 0.732, and a2 5 b2 5 1. Substituting these values into Eq. (29.24) yields the following balance for node (1, 1):

0.788675(T01 2 T11) 1 0.57735(T21 2 T11)

1 0.788675(T10 2 T11) 1 0.57735(T12 2 T11) 5 0

Collecting terms, we can express this equation as

24T11 1 0.8453T21 1 0.8453T12 5 21.1547T01 2 1.1547T10

cha9792x_ch29_852-872.indd Page 864 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 864 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.3 BOUNDARY CONDITIONS 865

The simultaneous equations for temperature distribution on the plate in Fig. 29.9 with a lower-edge boundary temperature of 75 can be written in matrix form as

4 20.845 20.845

21 4 21 21

21 4 21

21 4 21 21

21 21 4 21 21

21 21 4 21

21 4 21

21 21 4 21

21 21 24

   

T11 T21 T31 T12 T22 T32 T13 T23 T33

  5  

173.2

75

125

75

0

50

175

100

150

These equations can be solved for

T11 5 74.98  T21 5 72.76  T31 5 66.07

T12 5 74.23  T22 5 75.00  T32 5 66.52

T13 5 83.93  T23 5 83.48  T33 5 75.00

These results along with the computed fl uxes are displayed in Fig. 29.10. Note that the fl uxes are computed in the same fashion as in Sec. 29.2.3, with the exception that (a1 1 a2) and (b1 1 b2) are substituted for the 2’s in the denominators of Eqs. (29.14) and (29.15), respectively. Section 32.3 illustrates how this is done.

83.93 83.48 75.00

77.23 75.00 66.52

74.98 72.76 66.07

100�C

75�C

75�C 50�C

FIGURE 29.10 Temperature and fl ux distribution for a heated plate with a circular boundary.

cha9792x_ch29_852-872.indd Page 865 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 865 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

866 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

Derivative conditions for irregularly shaped boundaries are more diffi cult to formu- late. Figure 29.11 shows a point near an irregular boundary where the normal derivative is specifi ed. The normal derivative at node 3 can be approximated by the gradient between nodes 1 and 7,

0T 0h ` 3

5 T1 2 T7

L17 (29.25)

When u is less than 458 as shown, the distance from node 7 to 8 is Dx tan u, and linear interpolation can be used to estimate

T7 5 T8 1 (T6 2 T8) ¢x tan u ¢y

The length L17 is equal to Dxycos u. This length, along with the approximation for T7, can be substituted into Eq. (29.25) to give

T1 5 a ¢xcos ub 0T 0h ` 3 1 T6

¢x tan u ¢y

1 T8 a1 2 ¢x tan u¢y b (29.26) Such an equation provides a means for incorporating the normal gradient into the fi nite-difference approach. For cases where u is greater than 458, a different equation would be used. The determination of this formula will be left as a homework exercise.

29.4 THE CONTROL-VOLUME APPROACH

To summarize, the fi nite-difference or Taylor series approach divides the continuum into nodes (Fig. 29.12a). The underlying partial differential equation is written for each of these nodes. Finite-difference approximations are then substituted for the derivatives to convert the equations to an algebraic form.

�x

�y

8

7

6 5

1 4

3

2

FIGURE 29.11 A curved boundary where the normal gradient is specifi ed.

cha9792x_ch29_852-872.indd Page 866 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 866 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.4 THE CONTROL-VOLUME APPROACH 867

Such an approach is quite simple and straightforward for orthogonal (that is, rect- angular) grids and constant coeffi cients. However, the approach becomes a more diffi cult endeavor for derivative conditions on irregularly shaped boundaries. Figure 29.13 is an example of a system where additional diffi culties arise. This plate is made of two different materials and has unequal grid spacing. In addition, half of its top edge is subject to convective heat transfer, whereas half is insulated. Developing equations for node (4, 2) would require some additional derivation beyond the approaches developed to this point.

FIGURE 29.12 Two different perspectives for developing approximate solutions of PDEs: (a) fi nite-difference and (b) control-volume.

(a) Pointwise, finite-difference approach

(b) Control-volume approach

FIGURE 29.13 A heated plate with unequal grid spacing, two materials, and mixed boundary conditions.

Convection Insulated

In su

la te

d

(4, 2)

h h/2

h

(1, 1)

Material A Material B

�z

cha9792x_ch29_852-872.indd Page 867 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 867 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

868 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

The control-volume approach (also called the volume-integral approach) offers an alternative way to numerically approximate PDEs that is especially useful for cases such as Fig. 29.13. As in Fig. 29.12b, the approach resembles the pointwise approach in that points are determined across the domain. However, rather than approximating the PDE at a point, the approximation is applied to a volume surrounding the point. For an or- thogonal grid, the volume is formed by the perpendicular lines through the midpoint of each line joining adjacent nodes. A heat balance can then be developed for each volume in a fashion similar to Eq. (29.1). As an example, we will apply the control-volume approach to node (4, 2). First, the volume is defi ned by bisecting the lines joining the nodes. As in Fig. 29.14, the volume has conductive heat transfer through its left, right, and lower boundaries and convective heat transfer through half of its upper boundary. Notice that the transfer through the lower boundary involves both materials. A steady-state heat balance for the volume can be written in qualitative terms as

0 5 a left-side conduction

b 2 a right-side conduction

b 1 alower conduction material “a”

b 1 alower conduction

material “b” b 2 a upper

convection b (29.27)

Now the conduction fl ux rate can be represented by the fi nite-difference version of Fourier’s law. For example, for the left-side conduction gain, it would be

q 5 2k¿a T42 2 T41

h

where q has units of cal/cm2/s. This fl ux rate must be then multiplied by the area across which it enters (Dz 3 hy2) to give the rate of heat entering the volume per unit time,

Q 5 2k¿a T42 2 T41

h h

2 ¢z

where Q has units of cal/s.

h/2

h/2 h/4

4, 1 4, 2 4, 3

3, 2

FIGURE 29.14 A control volume for node (4, 2) with arrows indicating heat transfer through the boundaries.

cha9792x_ch29_852-872.indd Page 868 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 868 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

29.5 SOFTWARE TO SOLVE ELLIPTIC EQUATIONS 869

The heat fl ux due to convection can be formulated as

q 5 hc (Ta 2 T42)

where hc 5 a heat convection coeffi cient [cal/(s ? cm 2 ? 8C)] and Ta 5 the air temperature

(8C). Again, multiplication by the proper area yields the rate of heat fl ow per time,

Q 5 hc(Ta 2 T42) h

4 ¢z

The other transfers can be developed in a similar fashion and substituted into Eq. (29.27) to yield

0 5 2k¿a T42 2 T41

h h

2 ¢z 1 k¿b

T43 2 T42 hy2

h

2 ¢z

(left-side conduction)(right-side conduction)

2k¿a T42 2 T32

h h

2 ¢z 2 k¿b

T42 2 T32 h

h

4 ¢z 1 hc(Ta 2 T42)

h

4 ¢z

alower conductionb alower conductionb (upper convection) material “a” material “b”

Parameter values can then be substituted to yield the fi nal heat balance equation. For ex- ample, if Dz 5 0.5 cm, h 5 10 cm, k9a 5 0.3 cal/(s ? cm ? 8C), k9b 5 0.5 cal/(s ? cm ? 8C), and hc 5 0.1 cal/(s ? cm

2 ? 8C), the equation becomes

0.5875T42 2 0.075T41 2 0.25T43 2 0.1375T32 5 2.5

To make the equation comparable to the standard Laplacian, this equation can be mul- tiplied by 4y0.5875 so that the coeffi cient of the base node has a coeffi cient of 4,

4T42 2 0.510638T41 2 1.702128T43 2 0.93617T32 5 17.02128

For the standard cases covered to this point, the control-volume and pointwise fi nite- difference approaches yield identical results. For example, for node (1, 1) in Fig. 29.13, the balance would be

0 52k¿a T11 2 T01

h h ¢z 1 k¿a

T21 2 T11 h

h ¢z 2 k¿a T11 2 T10

h h ¢z 1 k¿a

T12 2 T11 h

h ¢z

which simplifi es to the standard Laplacian,

0 5 4T11 2 T01 2 T21 2 T12 2 T10

We will look at other standard cases (for example, the derivative boundary condition) and explore the control-volume approach in additional detail in the problems at the end of this chapter.

29.5 SOFTWARE TO SOLVE ELLIPTIC EQUATIONS

Modifying a computer program to include derivative boundary conditions for rectangular systems is a relatively straightforward task. It merely involves ensuring that additional equations are generated to characterize the boundary nodes at which the derivatives are specifi ed. In addition, the code must be modifi ed so that these equations incorporate the derivative as seen in Eq. (29.20).

cha9792x_ch29_852-872.indd Page 869 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 869 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

870 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

Developing general software to characterize systems with irregular boundaries is a much more diffi cult proposition. For example, a fairly involved algorithm would be re- quired to model the simple gasket depicted in Fig. 29.15. This would involve two major modifi cations. First, a scheme would have to be developed to conveniently input the con- fi guration of the nodes and to identify which were at the boundary. Second, an algorithm would be required to generate the proper simultaneous equations on the basis of the input information. The net result is that general software for solving elliptic (and for that matter, all) PDEs is relatively complicated. One method used to simplify such efforts is to require a very fi ne grid. For such cases, it is often assumed that the closest node serves as the boundary point. In this way, the analysis does not have to consider the weighting parameters from Sec. 29.3.2. Although this introduces some error, the use of a suffi ciently fi ne mesh can make the resulting dis- crepancy negligible. However, this involves a trade-off due to the computational burden introduced by the increased number of simultaneous equations. As a consequence of these considerations, numerical analysts have developed alterna- tive approaches that differ radically from fi nite-difference methods. Although these fi nite- element methods are more conceptually diffi cult, they can much more easily accommodate irregular boundaries. We will turn to these methods in Chap. 31. Before doing this, how- ever, we will fi rst describe fi nite-difference approaches for another category of PDEs— parabolic equations.

FIGURE 29.15 A fi nite-difference grid superimposed on an irregularly shaped gasket.

PROBLEMS

29.1 Use Liebmann’s method to solve for the temperature of the square heated plate in Fig. 29.4, but with the upper boundary condi- tion increased to 1508C and the left boundary insulated. Use a re- laxation factor of 1.2 and iterate to es 5 1%. 29.2 Use Liebmann’s method to solve for the temperature of the square heated plate in Fig. 29.4, but with the upper boundary condi-

tion increased to 1208C and the left boundary decreased to 608C. Use a relaxation factor of 1.2 and iterate to es 5 1%. 29.3 Compute the fl uxes for Prob. 29.2 using the parameters from Example 29.3. 29.4 Repeat Example 29.1, but use 49 interior nodes (that is, Dx 5 Dy 5 5 cm).

cha9792x_ch29_852-872.indd Page 870 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 870 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 871

29.5 Repeat Prob. 29.4, but for the case where the lower edge is insulated. 29.6 Repeat Examples 29.1 and 29.3, but for the case where the fl ux at the lower edge is directed downward with a value of 2 cal/cm2 ? s. 29.7 Repeat Example 29.4 for the case where both the lower left and the upper right corners are rounded in the same fashion as the lower left corner of Fig. 29.9. Note that all boundary temperatures on the upper and right sides are fi xed at 1008C and all on the lower and left sides are fi xed at 508C. 29.8 With the exception of the boundary conditions, the plate in Fig. P29.8 has the exact same characteristics as the plate used in Examples 29.1 through 29.3. Simulate both the temperatures and fl uxes for the plate. 29.9 Write equations for the darkened nodes in the grid in Fig. P29.9. Note that all units are cgs. The coeffi cient of thermal con- ductivity for the plate is 0.75 cal/(s ? cm ? 8C), the convection coef- fi cient is hc 5 0.015 cal/(cm

2 ? C ? s), and the thickness of the plate is 0.5 cm. 29.10 Write equations for the darkened nodes in the grid in Fig. P29.10. Note that all units are cgs. The convection coeffi- cient is hc 5 0.01 cal/(cm

2 ? C ? s) and the thickness of the plate is 2 cm. 29.11 Apply the control-volume approach to develop the equation for node (0, j) in Fig. 29.7. 29.12 Derive an equation like Eq. (29.26) for the case where u is greater than 458 for Fig. 29.11. 29.13 Develop a user-friendly computer program to implement Liebmann’s method for a rectangular plate with Dirichlet bound- ary conditions. Design the program so that it can compute both

temperature and fl ux. Test the program by duplicating the results of Examples 29.1 and 29.2. 29.14 Employ the program from Prob. 29.13 to solve Probs. 29.2 and 29.3. 29.15 Employ the program from Prob 29.13 to solve Prob. 29.4. 29.16 Use the control-volume approach and derive the node equa- tion for node (2, 2) in Fig. 29.13 and include a heat source at this point. Use the following values for the constants: Dz 5 0.25 cm, h 5 10 cm, kA 5 0.25 W/cm ? C, and kB 5 0.45 W/cm ? C. The heat source comes only from material A at the rate of 5 6 W/cm3. 29.17 Calculate heat fl ux (W/cm2) for node (2, 2) in Fig. 29.13 us- ing fi nite-difference approximations for the temperature gradients

0 25 50 75 100

75

50

25

0 Insulated

In su

la te

d

FIGURE P29.8

i = 0 1 2 3 4 5 Dirichlet, T = 100�C

0

1

2

j = 3

D ir

ic h

le t,

T =

5 0�

C

�y = 30 �x = 40

�x = 20

�y = 15

Convection, qy = –hc(Ta – T); Ta = 10�C

In su

la te

d

Heat source, qz = 10 cal/cm 2/s

FIGURE P29.9

i = 0 1 2 3 4 0

1

j = 2

� y

= 30�x = 40

�x = 20

� y

= 15

Insulated

k' = 0.7 k' = 0.5 qz = 10 cal/cm2/s

C o

n ve

ct io

n q x

= h

c( T

a –

T );

T a

= 20

�C

FIGURE P29.10

cha9792x_ch29_852-872.indd Page 871 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 871 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

872 FINITE DIFFERENCE: ELLIPTIC EQUATIONS

at this node. Calculate the fl ux in the horizontal direction in materi- als A and B, and determine if these two fl uxes should be equal. Also, calculate the vertical fl ux in materials A and B. Should these two fl uxes be equal? Use the following values for the constants: Dz 5 0.5 cm, h 5 10 cm, kA 5 0.25 W/cm ? C, kB 5 0.45 W/cm ? C, and nodal temperatures: T22 5 51.68C, T21 5 74.28C, T23 5 45.38C, T32 5 38.68C, and T12 5 87.48C. 29.18 Compute the temperature distribution for the L-shaped plate in Fig. P29.18. 29.19 The Poisson equation can be written in three dimensions as

02T 0x2

1 02T 0y2

1 02T 0z2

5 f (x, y, z)

Solve for the distribution of temperature within a unit (1 3 1) cube with zero boundary conditions and f 5 210. Employ Dx 5 Dy 5 Dz 5 1y6.

0

0

0

20

40

60

80

100

120

Insulated

Insulated

Insulated

FIGURE P29.18

cha9792x_ch29_852-872.indd Page 872 05/11/13 6:51 PM F-468 cha9792x_ch29_852-872.indd Page 872 05/11/13 6:51 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30

873

C H A P T E R 30

Finite Difference: Parabolic Equations

Chapter 29 dealt with steady-state PDEs. We now turn to the parabolic equations that are employed to characterize time-variable problems. In the latter part of this chapter, we will illustrate how this is done in two spatial dimensions for the heated plate. Before doing this, we will fi rst show how the simpler one-dimensional case is approached.

30.1 THE HEAT-CONDUCTION EQUATION

In a fashion similar to the derivation of the Laplace equation [Eq. (29.6)], conservation of heat can be used to develop a heat balance for the differential element in the long, thin insulated rod shown in Fig. 30.1. However, rather than examine the steady-state case, the present balance also considers the amount of heat stored in the element over a unit time period Dt. Thus, the balance is in the form, inputs 2 outputs 5 storage, or

q(x) ¢y ¢z ¢t 2 q(x 1 ¢x) ¢y ¢z ¢t 5 ¢x ¢y ¢zrC ¢T

Dividing by the volume of the element (5 Dx Dy Dz) and Dt gives

q(x) 2 q(x 1 ¢x) ¢x

5 rC ¢T ¢t

Taking the limit yields

2 0q 0x

5 rC 0T 0t

FIGURE 30.1 A thin rod, insulated at all points except at its ends.

CoolHot

cha9792x_ch30_873-889.indd Page 873 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 873 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

874 FINITE DIFFERENCE: PARABOLIC EQUATIONS

Substituting Fourier’s law of heat conduction [Eq. (29.4)] results in

k 02T 0x2

5 0T 0t

(30.1)

which is the heat-conduction equation. Just as with elliptic PDEs, parabolic equations can be solved by substituting fi nite divided differences for the partial derivatives. However, in contrast to elliptic PDEs, we must now consider changes in time as well as in space. Whereas elliptic equations were bounded in all relevant dimensions, parabolic PDEs are temporally open-ended (Fig. 30.2). Because of their time-variable nature, solutions to these equations involve a number of new issues, notably stability. This, as well as other aspects of parabolic PDEs, will be examined in the following sections as we present two fundamental solution approaches— explicit and implicit schemes.

30.2 EXPLICIT METHODS

The heat-conduction equation requires approximations for the second derivative in space and the fi rst derivative in time. The former is represented in the same fashion as for the Laplace equation by a centered fi nite-divided difference:

02T 0x2

5 T li11 2 2T

l i 1 T

l i21

¢x2 (30.2)

FIGURE 30.2 A grid used for the fi nite-difference solution of parabolic PDEs in two independent variables such as the heat-conduction equation. Note how, in contrast to Fig. 29.3, this grid is open-ended in the temporal dimension.

t

x

i – 1, l i + 1, l

m + 1, 0

i, l – 1

i, l + 1

0, 0

i, l

cha9792x_ch30_873-889.indd Page 874 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 874 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.2 EXPLICIT METHODS 875

which has an error (recall Fig. 23.3) of O[(Dx)2]. Notice the slight change in notation of the superscripts is used to denote time. This is done so that a second subscript can be used to designate a second spatial dimension when the approach is expanded to two spatial dimensions. A forward fi nite-divided difference is used to approximate the time derivative

0T 0t

5 Tl11i 2 T

l i

¢t (30.3)

which has an error (recall Fig. 23.1) of O(Dt). Substituting Eqs. (30.2) and (30.3) into Eq. (30.1) yields

k Tli11 2 2T

l i 1 T

l i21

(¢x)2 5

Tl11i 2 T l i

¢t (30.4)

which can be solved for

Tl11i 5 T l i 1 l(Tli11 2 2Tli 1 Tli21) (30.5)

where l 5 k Dty(Dx)2. This equation can be written for all the interior nodes on the rod. It then provides an explicit means to compute values at each node for a future time based on the present values at the node and its neighbors. Notice that this approach is actually a manifestation of Euler’s method for solving systems of ODEs. That is, if we know the temperature distribution as a function of position at an initial time, we can compute the distribution at a future time based on Eq. (30.5). A computational molecule for the explicit method is depicted in Fig. 30.3, show- ing the nodes that constitute the spatial and temporal approximations. This molecule can be contrasted with others in this chapter to illustrate the differences between approaches.

FIGURE 30.3 A computational molecule for the explicit form.

Grid point involved in time difference

Grid point involved in space difference

xi – 1 xi xi + 1

t l

t l + 1

cha9792x_ch30_873-889.indd Page 875 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 875 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

876 FINITE DIFFERENCE: PARABOLIC EQUATIONS

EXAMPLE 30.1 Explicit Solution of the One-Dimensional Heat-Conduction Equation

Problem Statement. Use the explicit method to solve for the temperature distribution of a long, thin rod with a length of 10 cm and the following values: k9 5 0.49 cal/(s ? cm ? 8C), Dx 5 2 cm, and Dt 5 0.1 s. At t 5 0, the temperature of the rod is zero and the bound- ary conditions are fi xed for all times at T(0) 5 1008C and T(10) 5 508C. Note that the rod is aluminum with C 5 0.2174 cal/(g ? 8C) and r 5 2.7 g/cm3. Therefore, k 5 0.49/ (2.7 ? 0.2174) 5 0.835 cm2/s and l 5 0.835(0.1)y(2)2 5 0.020875.

Solution. Applying Eq. (30.5) gives the following value at t 5 0.1 s for the node at x 5 2 cm:

T 11 5 0 1 0.020875[0 2 2(0) 1 100] 5 2.0875

At the other interior points, x 5 4, 6, and 8 cm, the results are

T 12 5 0 1 0.020875[0 2 2(0) 1 0] 5 0

T 13 5 0 1 0.020875[0 2 2(0) 1 0] 5 0

T 14 5 0 1 0.020875[50 2 2(0) 1 0] 5 1.0438

At t 5 0.2 s, the values at the four interior nodes are computed as

T 21 5 2.0875 1 0.020875[0 2 2(2.0875) 1 100] 5 4.0878

T 22 5 0 1 0.020875[0 2 2(0) 1 2.0875] 5 0.043577

T 23 5 0 1 0.020875[1.0438 2 2(0) 1 0] 5 0.021788

T 24 5 1.0438 1 0.020875[50 2 2(1.0438) 1 0] 5 2.0439

The computation is continued, and the results at 3-s intervals are depicted in Fig. 30.4. The general rise in temperature with time indicates that the computation captures the diffusion of heat from the boundaries into the bar.

FIGURE 30.4 Temperature distribution in a long, thin rod as computed with the explicit method described in Sec. 30.2.

T

40

80

0 4 8 x

� t = 0.1 �x = 2 k = 0.835

t = 12

t = 9

t = 6t = 3

cha9792x_ch30_873-889.indd Page 876 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 876 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.2 EXPLICIT METHODS 877

30.2.1 Convergence and Stability

Convergence means that as Dx and Dt approach zero, the results of the fi nite-difference technique approach the true solution. Stability means that errors at any stage of the computation are not amplifi ed but are attenuated as the computation progresses. It can be shown (Carnahan et al., 1969) that the explicit method is both convergent and stable if l # 1y2, or

¢t # 1

2 ¢x2

k (30.6)

In addition, it should be noted that setting l # 1y2 could result in a solution in which errors do not grow, but oscillate. Setting l # 1y4 ensures that the solution will not oscillate. It is also known that setting l 5 1y6 tends to minimize truncation error (Carnahan et al., 1969). Figure 30.5 is an example of instability caused by violating Eq. (30.6). This plot is for the same case as in Example 30.1 but with l 5 0.735, which is considerably greater than 0.5. As in Fig. 30.5, the solution undergoes progressively increasing oscillations. This situation will continue to deteriorate as the computation continues. Although satisfaction of Eq. (30.6) will alleviate the instabilities of the sort mani- fested in Fig. 30.5, it also places a strong limitation on the explicit method. For example, suppose that Dx is halved to improve the approximation of the spatial second derivative. According to Eq. (30.6), the time step must be quartered to maintain convergence and stability. Thus, to perform comparable computations, the time steps must be increased by a factor of 4. Furthermore, the computation for each of these time steps will take twice as long because halving Dx doubles the total number of nodes for which equations must be written. Consequently, for the one-dimensional case, halving Dx results in an eightfold increase in the number of calculations. Thus, the computational burden may be large to attain acceptable accuracy. As will be described shortly, other techniques are available that do not suffer from such severe limitations.

30.2.2 Derivative Boundary Conditions

As was the case for elliptic PDEs (recall Sec. 29.3.1), derivative boundary conditions can be readily incorporated into parabolic equations. For a one-dimensional rod, this necessitates adding two equations to characterize the heat balance at the end nodes. For example, the node at the left end (i 5 0) would be represented by

Tl110 5 T l 0 1 l(T l1 2 2Tl0 1 Tl21)

Thus, an imaginary point is introduced at i 5 21 (recall Fig. 29.7). However, as with the elliptic case, this point provides a vehicle for incorporating the derivative boundary condi- tion into the analysis. Problem 30.2 at the end of the chapter deals with this exercise.

30.2.3 Higher-Order Temporal Approximations

The general idea of reexpressing the PDE as a system of ODEs is sometimes called the method of lines. Obviously, one way to improve on the Euler approach used above would be to employ a more accurate integration scheme for solving the ODEs. For example, the Heun method can be employed to obtain second-order temporal accuracy. An example of

cha9792x_ch30_873-889.indd Page 877 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 877 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

878 FINITE DIFFERENCE: PARABOLIC EQUATIONS

this approach is called MacCormack’s method. This and other improved explicit methods are discussed elsewhere (for example, Hoffman, 1992).

30.3 A SIMPLE IMPLICIT METHOD

As noted previously, explicit fi nite-difference formulations have problems related to stabil- ity. In addition, as depicted in Fig. 30.6, they exclude information that has a bearing on the solution. Implicit methods overcome both these diffi culties at the expense of somewhat more complicated algorithms.

FIGURE 30.5 An illustration of instability. Solution of Example 30.1 but with l 5 0.735.

T

100

0 x

t = 6

T

100

0 x

t = 12

T

100

0 x

t = 18

T

100

0 x

t = 24

T

100

0 0 4 8 x

t = 30

cha9792x_ch30_873-889.indd Page 878 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 878 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.3 A SIMPLE IMPLICIT METHOD 879

The fundamental difference between explicit and implicit approximations is depicted in Fig. 30.7. For the explicit form, we approximate the spatial derivative at time level l (Fig. 30.7a). Recall that when we substituted this approximation into the partial differ- ential equation, we obtained a difference equation (30.4) with a single unknown Tl11i . Thus, we can solve “explicitly” for this unknown as in Eq. (30.5). In implicit methods, the spatial derivative is approximated at an advanced time level l 1 1. For example, the second derivative would be approximated by (Fig. 30.7b)

02T 0x2

> Tl11i11 2 2T

l11 i 1 T

l11 i21

(¢x)2 (30.7)

which is second-order accurate. When this relationship is substituted into the original PDE, the resulting difference equation contains several unknowns. Thus, it cannot be solved explicitly by simple algebraic rearrangement as was done in going from Eq. (30.4)

FIGURE 30.6 Representation of the effect of other nodes on the fi nite- difference approximation at node (i, I) using an explicit fi nite-difference scheme. The shaded nodes have an infl uence on (i, I), whereas the unshaded nodes, which in reality affect (i, I), are excluded.

t

x Initial condition

Boundary condition

(i, l)

FIGURE 30.7 Computational molecules demonstrating the fundamental differences between (a) explicit and (b) implicit methods.

Grid point involved in time difference

Grid point involved in space difference

i – 1 i i + 1

l

l + 1

(a) Explicit

i – 1 i i + 1

l

l + 1

(b) Implicit

cha9792x_ch30_873-889.indd Page 879 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 879 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

880 FINITE DIFFERENCE: PARABOLIC EQUATIONS

to (30.5). Instead, the entire system of equations must be solved simultaneously. This is possible because, along with the boundary conditions, the implicit formulations result in a set of linear algebraic equations with the same number of unknowns. Thus, the method reduces to the solution of a set of simultaneous equations at each point in time. To illustrate how this is done, substitute Eqs. (30.3) and (30.7) into Eq. (30.1) to give

k Tl11i11 2 2T

l11 i 1 T

l11 i21

(¢x)2 5

Tl11i 2 T l i

¢t

which can be expressed as

2lTl11i21 1 (1 1 2l)T l11 i 2 lT

l11 i11 5 T

l i (30.8)

where l 5 k Dty(Dx)2. This equation applies to all but the fi rst and the last interior nodes, which must be modifi ed to refl ect the boundary conditions. For the case where the tem- perature levels at the ends of the rod are given, the boundary condition at the left end of the rod (i 5 0) can be expressed as

Tl110 5 f0(t l11) (30.9)

where f0(t l11) 5 a function describing how the boundary temperature changes with time. Substituting Eq. (30.9) into Eq. (30.8) gives the difference equation for the fi rst interior node (i 5 1):

(1 1 2l)Tl111 2 lT l11 2 5 T

l 1 1 l f0(t l11) (30.10)

Similarly, for the last interior node (i 5 m),

2lTl11m21 1 (1 1 2l)T l11 m 5 T

l m 1 l fm11(t l11) (30.11)

where fm11(t l11) describes the specifi ed temperature changes at the right end of the rod (i 5 m 1 1). When Eqs. (30.8), (30.10), and (30.11) are written for all the interior nodes, the resulting set of m linear algebraic equations has m unknowns. In addition, the method has the added bonus that the system is tridiagonal. Thus, we can utilize the extremely effi cient solution algorithms (recall Sec. 11.1.1) that are available for tridiagonal systems.

EXAMPLE 30.2 Simple Implicit Solution of the Heat-Conduction Equation

Problem Statement. Use the simple implicit fi nite-difference approximation to solve the same problem as in Example 30.1.

Solution. For the rod from Example 30.1, l 5 0.020875. Therefore, at t 5 0, Eq. (30.10) can be written for the fi rst interior node as

1.04175T 11 2 0.020875T 1 2 5 0 1 0.020875(100)

or

1.04175T 11 2 0.020875T 1 2 5 2.0875

cha9792x_ch30_873-889.indd Page 880 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 880 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.3 A SIMPLE IMPLICIT METHOD 881

In a similar fashion, Eqs. (30.8) and (30.11) can be applied to the other interior nodes. This leads to the following set of simultaneous equations:

≥ 1.04175 20.020875

20.020875 1.04175 20.020875

20.020875 1.04175 20.020875

20.020875 1.04175

¥ μ T 1

1

T 2 1

T 3 1

T 4 1

∂5 μ 2.0875

0

0

1.04375

which can be solved for the temperature at t 5 0.1 s:

T 11 5 2.0047

T 12 5 0.0406

T 13 5 0.0209

T 14 5 1.0023

Notice how in contrast to Example 30.1, all the points have changed from the initial con- dition during the fi rst time step. To solve for the temperatures at t 5 0.2, the right-hand-side vector must be modifi ed to account for the results of the fi rst step, as in

μ 4.09215

0.04059

0.02090

2.04069

The simultaneous equations can then be solved for the temperatures at t 5 0.2 s:

T 21 5 3.9305

T 22 5 0.1190

T 23 5 0.0618

T 24 5 1.9653

Whereas the implicit method described is stable and convergent, it has the defect that the temporal difference approximation is fi rst-order accurate, whereas the spatial difference approximation is second-order accurate (Fig. 30.8). In the next section we present an alternative implicit method that remedies the situation. Before proceeding, it should be mentioned that, although the simple implicit method is unconditionally stable, there is an accuracy limit to the use of large time steps. Con- sequently, it is not that much more effi cient than the explicit approaches for most time- variable problems. Where it does shine is for steady-state problems. Recall from Chap. 29 that a form of Gauss-Seidel (Liebmann’s method) can be used to obtain steady-state solutions for elliptic equations. An alternative approach would be to run a time-variable solution until it reached a steady state. In such cases, because inaccurate intermediate results are not an issue, implicit methods allow you to employ larger time steps, and hence, can generate steady-state results in an effi cient manner.

cha9792x_ch30_873-889.indd Page 881 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 881 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

882 FINITE DIFFERENCE: PARABOLIC EQUATIONS

30.4 THE CRANK-NICOLSON METHOD

The Crank-Nicolson method provides an alternative implicit scheme that is second-order accurate in both space and time. To provide this accuracy, difference approximations are developed at the midpoint of the time increment (Fig. 30.9). To do this, the temporal fi rst derivative can be approximated at tl11y2 by

0T 0t

> Tl11i 2 T

l i

¢t (30.12)

The second derivative in space can be determined at the midpoint by averaging the dif- ference approximations at the beginning (t l) and at the end (t l11) of the time increment

02T 0x2

> 1

2 c Tli11 2 2Tli 1 Tli11

(¢x)2 1

Tl11i11 2 2T l11 i 1 T

l11 i21

(¢x)2 d (30.13)

FIGURE 30.8 A computational molecule for the simple implicit method.

Grid point involved in time difference

Grid point involved in space difference

xi – 1 xi xi + 1

t l

t l + 1

FIGURE 30.9 A computational molecule for the Crank-Nicolson method.

Grid point involved in time difference

Grid point involved in space difference

xi – 1 xi xi + 1

t l

t l + 1

t l + 1/2

cha9792x_ch30_873-889.indd Page 882 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 882 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.4 THE CRANK-NICOLSON METHOD 883

Substituting Eqs. (30.12) and (30.13) into Eq. (30.1) and collecting terms gives

2lTl11i21 1 2(1 1 l)T l11 i 2 lT

l11 i11 5 lT

l i21 1 2(1 2 l)T

l i 1 lT

l i11 (30.14)

where l 5 k Dty(Dx)2. As was the case with the simple implicit approach, boundary conditions of Tl110 5 f0(t l11) and Tl11m11 5 fm11(t l11) can be prescribed to derive versions of Eq. (30.14) for the fi rst and the last interior nodes. For the fi rst interior node

2(1 1 l)Tl111 2 lT l11 2 5 l f0(t l) 1 2(1 2 l)Tl1 1 lTl2 1 l f0(t l11) (30.15)

and for the last interior node,

2lTl11m21 1 2(1 1 l)T l11 m 5 l fm11(t l) 1 2(1 2 l)Tlm 1 lTlm21 1 l fm11(t l11)

(30.16)

Although Eqs. (30.14) through (30.16) are slightly more complicated than Eqs. (30.8), (30.10), and (30.11), they are also tridiagonal and, therefore, effi cient to solve.

EXAMPLE 30.3 Crank-Nicolson Solution to the Heat-Conduction Equation

Problem Statement. Use the Crank-Nicolson method to solve the same problem as in Examples 30.1 and 30.2.

Solution. Equations (30.14) through (30.16) can be employed to generate the following tridiagonal set of equations:

≥ 2.04175 20.020875

20.020875 2.04175 20.020875

20.020875 2.04175 20.020875

20.020875 2.04175

¥ μ T 1

1

T 2 1

T 3 1

T 4 1

∂5 μ 4.175

0

0

2.0875

which can be solved for the temperatures at t 5 0.1 s:

T 11 5 2.0450

T 12 5 0.0210

T 13 5 0.0107

T 14 5 1.0225

To solve for the temperatures at t 5 0.2 s, the right-hand-side vector must be changed to

μ 8.1801

0.0841

0.0427

4.0901

The simultaneous equations can then be solved for

T 21 5 4.0073

T 22 5 0.0826

T 23 5 0.0422

T 24 5 2.0036

cha9792x_ch30_873-889.indd Page 883 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 883 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

884 FINITE DIFFERENCE: PARABOLIC EQUATIONS

30.4.1 Comparison of One-Dimensional Methods

Equation (30.1) can be solved analytically. For example, a solution is available for the case where the rod’s temperature is initially at zero. At t 5 0, the boundary condition at x 5 L is instantaneously increased to a constant level of T while T(0) is held at zero. For this case, the temperature can be computed by

T 5 T c x L

1 a q

n50

2 np

(21)n sin anx L b exp a2n2p2kt

L2 b d (30.17)

where L 5 total length of the rod. This equation can be employed to compute the evolu- tion of the temperature distribution for each boundary condition. Then, the total solution can be determined by superposition.

EXAMPLE 30.4 Comparison of Analytical and Numerical Solutions

Problem Statement. Compare the analytical solution from Eq. (30.17) with numerical results obtained with the explicit, simple implicit, and Crank-Nicolson techniques. Per- form the comparison for the rod employed in Examples 30.1, 30.2, and 30.3.

Solution. Recall from the previous examples that k 5 0.835 cm2/s, L 5 10 cm, and Dx 5 2 cm. For this case, Eq. (30.17) can be used to predict that the temperature at x 5 2 cm, and t 5 10 s would equal 64.8018. Table 30.1 presents numerical predictions of T(2, 10). Notice that a range of time steps are employed. These results indicate a number of properties of the numerical methods. First, it can be seen that the explicit method is unstable for high values of l. This instability is not manifested by either implicit approach. Second, the Crank-Nicolson method converges more rapidly as l is decreased and provides moderately accurate results even when l is relatively high. These outcomes are as expected because Crank-Nicolson is second-order accurate with respect to both independent variables. Finally, notice that as l decreases, the methods seem to be converg- ing on a value of 64.73 that is different than the analytical result of 64.80. This should not be surprising because a fi xed value of Dx 5 2 is used to characterize the x dimension. If both Dx and Dt were decreased as l was decreased (that is, more spatial segments were used), the numerical solution would more closely approach the analytical result.

TABLE 30.1 Comparison of three methods of solving a parabolic PDE: the heated rod. The results shown are for temperature at t 5 10 s at x 5 2 cm for the rod from Examples 30.1 through 30.3. Note that the analytical solution is T (2, 10) 5 64.8018.

Dt L Explicit Implicit Crank-Nicolson

10 2.0875 208.75 53.01 79.77 5 1.04375 29.13 58.49 64.79 2 0.4175 67.12 62.22 64.87 1 0.20875 65.91 63.49 64.77 0.5 0.104375 65.33 64.12 64.74 0.2 0.04175 64.97 64.49 64.73

cha9792x_ch30_873-889.indd Page 884 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 884 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.5 PARABOLIC EQUATIONS IN TWO SPATIAL DIMENSIONS 885

The Crank-Nicolson method is often used for solving linear parabolic PDEs in one spatial dimension. Its advantages become even more pronounced for more complicated applications such as those involving unequally spaced meshes. Such nonuniform spacing is often advantageous where we have foreknowledge that the solution varies rapidly in local portions of the system. Further discussion of such applications and the Crank-Nicolson method in general can be found elsewhere (Ferziger, 1981; Lapidus and Pinder, 1981; Hoffman, 1992).

30.5 PARABOLIC EQUATIONS IN TWO SPATIAL DIMENSIONS

The heat-conduction equation can be applied to more than one spatial dimension. For two dimensions, its form is

0T 0t

5 k a 02T 0x2

1 02T 0y2 b (30.18)

One application of this equation is to model the temperature distribution on the face of a heated plate. However, rather than characterizing its steady-state distribution, as was done in Chap. 29, Eq. (30.18) provides a means to compute the plate’s temperature distribution as it changes in time.

30.5.1 Standard Explicit and Implicit Schemes

An explicit solution can be obtained by substituting fi nite-difference approximations of the form of Eqs. (30.2) and (30.3) into Eq. (30.18). However, as with the one-dimensional case, this approach is limited by a stringent stability criterion. For the two-dimensional case, the criterion is

¢t # 1

8 (¢x)2 1 (¢y)2

k

Thus, for a uniform grid (Dx 5 Dy), l 5 kDty(Dx)2 must be less than or equal to 1y4. Consequently, halving the step size results in a fourfold increase in the number of nodes and a 16-fold increase in computational effort. As was the case with one-dimensional systems, implicit techniques offer alternatives that guarantee stability. However, the direct application of implicit methods such as the Crank-Nicolson technique leads to the solution of m 3 n simultaneous equations. Addi- tionally, when written for two or three spatial dimensions, these equations lose the valu- able property of being tridiagonal. Thus, matrix storage and computation time can become exorbitantly large. The method described in the next section offers one way around this dilemma.

30.5.2 The ADI Scheme

The alternating-direction implicit, or ADI, scheme provides a means for solving parabolic equations in two spatial dimensions using tridiagonal matrices. To do this, each time

cha9792x_ch30_873-889.indd Page 885 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 885 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

886 FINITE DIFFERENCE: PARABOLIC EQUATIONS

increment is executed in two steps (Fig. 30.10). For the fi rst step, Eq. (30.18) is ap- proximated by

Tl11y2i, j 2 T l i, j

¢ty2 5 k c T

l i11, j 2 2T

l i, j 1 T

l i21, j

(¢x)2 1

Tl11y2i, j11 2 2T l11y2 i, j 1 T

l11y2 i, j21

(¢y)2 d (30.19)

Thus, the approximation of 02Ty0x2 is written explicitly—that is, at the base point tl where values of temperature are known. Consequently, only the three temperature terms in the approximation of 02Ty0y2 are unknown. For the case of a square grid (Dy 5 Dx), this equation can be expressed as

2lTl11y2i, j21 1 2(1 1 l)T l11y2 i, j 2 lT

l11y2 i, j11 5 lT

l i21, j 1 2(1 2 l)T

l i, j 1 lT

l i11, j

(30.20)

which, when written for the system, results in a tridiagonal set of simultaneous equations. For the second step from t l11y2 to t l11, Eq. (30.18) is approximated by

Tl11i, j 2 T l11y2 i, j

¢ty2 5 k c T

l11 i11, j 2 2T

l11 i, j 1 T

l11 i21, j

(¢x)2 1

Tl11y2i, j11 2 2T l11y2 i, j 1 T

l11y2 i, j21

(¢y)2 d

(30.21)

In contrast to Eq. (30.19), the approximation of 02Ty0x2 is now implicit. Thus, the bias introduced by Eq. (30.19) will be partially corrected. For a square grid, Eq. (30.21) can be written as

2lTl11i21, j 1 2(1 1 l)T l11 i, j 2 lT

l11 i11, j 5 lT

l11y2 i, j21 1 2(1 2 l)T

l11y2 i, j 1 lT

l11y2 i, j11

(30.22)

Again, when written for a two-dimensional grid, the equation results in a tridiagonal system (Fig. 30.11). As in the following example, this leads to an effi cient numerical solution.

FIGURE 30.10 The two half-steps used in implementing the alternating- direction implicit scheme for solving parabolic equations in two spatial dimensions.

yj + 1

yj – 1 xi – 1 xi xi + 1

t l + 1

t l + 1/2

t l

xi – 1 xi xi + 1

yj

yj + 1

yj – 1

yj

Explicit Implicit

(a) First half-step (b) Second half-step

cha9792x_ch30_873-889.indd Page 886 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 886 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

30.5 PARABOLIC EQUATIONS IN TWO SPATIAL DIMENSIONS 887

FIGURE 30.11 The ADI method only results in tridiagonal equations if it is applied along the dimension that is implicit. Thus, on the fi rst step (a), it is applied along the y dimension and, on the second step (b), along the x dimension. These “alternating directions” are the root of the method’s name.

(a) First direction (b) Second direction

i = 1 i = 1i = 2 i = 2

j = 3

j = 2

j = 1

i = 3 i = 3

y

x

EXAMPLE 30.5 ADI Method

Problem Statement. Use the ADI method to solve for the temperature of the plate in Examples 29.1 and 29.2. At t 5 0, assume that the temperature of the plate is zero and the boundary temperatures are instantaneously brought to the levels shown in Fig. 29.4. Employ a time step of 10 s. Recall from Example 30.1 that the coeffi cient of thermal dif- fusivity for aluminum is k 5 0.835 cm2/s.

Solution. A value of Dx 5 10 cm was employed to characterize the 40 3 40-cm plate from Examples 29.1 and 29.2. Therefore, l 5 0.835(10)y(10)2 5 0.0835. For the fi rst step to t 5 5 (Fig. 30.11a), Eq. (30.20) is applied to nodes (1, 1), (1, 2), and (1, 3) to yield the following tridiagonal equations:

£ 2.167 20.0835

20.0835 2.167 20.0835

20.0835 2.167

§ • T1,1 T1,2 T1,3

¶ 5 • 6.2625

6.2625

14.6125

¶ which can be solved for

T1,1 5 3.01597  T1, 2 5 3.2708  T1, 3 5 6.8692

In a similar fashion, tridiagonal equations can be developed and solved for

T2,1 5 0.1274  T2, 2 5 0.2900  T2, 3 5 4.1291 and

T3,1 5 2.0181  T3, 2 5 2.2477  T3, 3 5 6.0256

For the second step to t 5 10 (Fig. 30.11b), Eq. (30.22) is applied to nodes (1, 1), (2, 1), and (3, 1) to yield

£ 2.167 20.0835

20.0835 2.167 20.0835

20.0835 2.167

§ • T1,1 T2,1 T3,1

¶ 5 • 12.0639

0.2577

8.0619

cha9792x_ch30_873-889.indd Page 887 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 887 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

888 FINITE DIFFERENCE: PARABOLIC EQUATIONS

which can be solved for

T1,1 5 5.5855  T2,1 5 0.4782  T3,1 5 3.7388

Tridiagonal equations for the other rows can be developed and solved for

T1, 2 5 6.1683  T2, 2 5 0.8238  T3, 2 5 4.2359

and

T1, 3 5 13.1120  T2, 3 5 8.3207  T3, 3 5 11.3606

The computation can be repeated, and the results for t 5 100, 200, and 300 s are depicted in Fig. 30.12a through c, respectively. As expected, the temperature of the plate rises. After a suffi cient time elapses, the temperature will approach the steady-state dis- tribution of Fig. 29.5.

The ADI method is but one of a group of techniques called splitting methods. Some of these represent efforts to circumvent shortcomings of ADI. Discussion of other splitting methods as well as more information on ADI can be found elsewhere (Ferziger, 1981; Lapidus and Pinder, 1981).

FIGURE 30.12 Solution for the heated plate from Example 30.5 at (a) t 5 100 s, (b) t 5 200 s, and (c) t 5 300 s.

28.56 14.57 20.73

41.09 27.20 31.94

60.76 52.57 53.02

(a) t = 100 s

37.40 25.72 28.69

55.26 45.32 44.86

72.82 68.17 64.12

(b) t = 200 s

40.82 30.43 31.96

60.30 52.25 49.67

76.54 73.29 67.68

(c) t = 300 s

PROBLEMS

30.1 Repeat Example 30.1, but use the midpoint method to gener- ate your solution. 30.2 Repeat Example 30.1, but for the case where the rod is ini- tially at 508C and the derivative at x 5 0 is equal to 1 and at x 5 10 is equal to 0. Interpret your results. 30.3 (a) Repeat Example 30.1, but for a time step of Dt 5 0.05 s. Compute results to t 5 0.2. (b) In addition, perform the same com- putation with the Heun method (without iteration of the corrector)

with a much smaller step size of Dt 5 0.001 s. Assuming that the results of (b) are a valid approximation of the true solution, deter- mine percent relative errors for the results obtained in Example 30.1 as well as for part (a). 30.4 Repeat Example 30.2, but for the case where the derivative at x 5 10 is equal to zero. 30.5 Repeat Example 30.3, but for Dx 5 1 cm. 30.6 Repeat Example 30.5, but for the plate described in Prob. 29.2.

cha9792x_ch30_873-889.indd Page 888 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 888 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 889

program to obtain the solution. Increase the value of ¢t by 10% for each time step to more quickly obtain the steady-state solution, and select values of ¢x and ¢t for good accuracy. Plot the nondimen- sional temperature versus nondimensional length for various values of nondimensional times. 30.14 The problem of transient radial heat fl ow in a circular rod in nondimensional form is described by

02u 0r 2

1 1

r 0u 0r

5 0u t

Boundary conditions u(1, t ) 5 1 0u 0t

(0, t) 5 0

Initial conditions u(x, 0) 5 0 0 # x # 1

Solve the nondimensional transient radial heat-conduction equa- tion in a circular rod for the temperature distribution at various times as the rod temperature approaches steady state. Use second- order accurate fi nite-difference analogues for the derivatives with a Crank-Nicolson formulation. Write a computer program for the solution. Select values of ¢r and ¢t for good accuracy. Plot the temperature u versus radius r for various times t. 30.15 Solve the following PDE:

02u 0x2

1 b 0u 0x

5 0u 0t

Boundary conditions u (0, t) 5 0 u (1, t ) 5 0 Initial conditions u (x, 0) 5 0 0 # x # 1

Use second-order accurate fi nite-difference analogues for the deriva- tives with a Crank-Nicolson formulation to integrate in time. Write a computer program for the solution. Increase the value of Dt by 10% for each time step to more quickly obtain the steady-state solution, and select values of Dx and Dt for good accuracy. Plot u versus x for various values of t. Solve for values of b 5 4, 2, 0, 22, 24. 30.16 Determine the temperatures along a 1-m horizontal rod de- scribed by the heat-conduction equation (Eq. 30.1). Assume that the right boundary is insulated and that the left boundary (x 5 0) is represented by

2k¿ 0T 0x ` x50

5 h(Ta 2 T0)

where k9 5 coeffi cient of thermal conductivity (W/m ? 8C), h 5 convective heat transfer coeffi cient (W/m2 ? 8C), Ta 5 ambient temperature (8C), and T0 5 temperature of the rod at x 5 0 (8C). Solve for temperature as a function of time using a spatial step of Dx 5 1 cm and the following parameter values: k 5 2 3 1025 m2/s, k9 5 10 W/m ? 8C, h 5 25 W/m2 ? 8C, and Ta 5 50 8C. Assume that the initial temperature of the rod is zero.

30.7 The advection-diffusion equation is used to compute the dis- tribution of concentration along the length of a rectangular chemi- cal reactor (see Sec. 32.1),

0c 0t

5 D 02c 0x2

2 U 0c 0x

2 kc

where c 5 concentration (mg/m3), t 5 time (min), D 5 a diffusion coeffi cient (m2/min), x 5 distance along the tank’s longitudinal axis (m) where x 5 0 at the tank’s inlet, U 5 velocity in the x direction (m/min), and k 5 a reaction rate (min21) whereby the chemical de- cays to another form. Develop an explicit scheme to solve this equa- tion numerically. Test it for k 5 0.15, D 5 100, and U 5 1 for a tank of length 10 m. Use a Dx 5 1 m, and a step size Dt 5 0.005. Assume that the infl ow concentration is 100 and that the initial concentration in the tank is zero. Perform the simulation from t 5 0 to 100 and plot the fi nal resulting concentrations versus x. 30.8 Develop a user-friendly computer program for the simple ex- plicit method from Sec. 30.2. Test it by duplicating Example 30.1. 30.9 Modify the program in Prob. 30.8 so that it employs either Dirichlet or derivative boundary conditions. Test it by solving Prob. 30.2. 30.10 Develop a user-friendly computer program to implement the simple implicit scheme from Sec. 30.3. Test it by duplicating Example 30.2. 30.11 Develop a user-friendly computer program to implement the Crank-Nicolson method from Sec. 30.4. Test it by duplicating Example 30.3. 30.12 Develop a user-friendly computer program for the ADI method described in Sec. 30.5. Test it by duplicating Example 30.5. 30.13 The nondimensional form for the transient heat conduction in an insulated rod (Eq. 30.1) can be written as

02u 0x  2

5 0u 0t

where nondimensional space, time, and temperature are defi ned as

x 5 x

L   t 5

T

(rCL2yk)  u 5

T 2 To TL 2 To

where L 5 the rod length, k 5 thermal conductivity of the rod ma- terial, r 5 density, C 5 specifi c heat, To 5 temperature at x 5 0, and TL 5 temperature at x 5 L. This makes for the following boundary and initial conditions:

Boundary conditions u(0, t ) 5 0 u(1, t ) 5 0 Initial conditions u(x,0) 5 0 0 # x # 1

Solve this nondimensional equation for the temperature distribu- tion using fi nite-difference methods and a second-order accurate Crank-Nicolson formulation to integrate in time. Write a computer

cha9792x_ch30_873-889.indd Page 889 05/11/13 6:53 PM F-468 cha9792x_ch30_873-889.indd Page 889 05/11/13 6:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31 C H A P T E R 31

890

Finite-Element Method

To this juncture, we have employed fi nite-difference methods to solve partial differential equations. In these methods, the solution domain is divided into a grid of discrete points or nodes (Fig. 31.1b). The PDE is then written for each node and its derivatives replaced by fi nite-divided differences. Although such “pointwise” approximation is conceptually easy to understand, it has a number of shortcomings. In particular, it becomes harder to apply for systems with irregular geometry, unusual boundary conditions, or heterogenous composition. The fi nite-element method provides an alternative that is better suited for such systems. In contrast to fi nite-difference techniques, the fi nite-element method divides the solution domain into simply shaped regions, or “elements” (Fig. 31.1c). An approximate solution for

FIGURE 31.1 (a) A gasket with irregular geometry and nonhomogeneous composition. (b) Such a system is very diffi cult to model with a fi nite-difference approach. This is due to the fact that complicated approx- imations are required at the boundaries of the system and at the boundaries between regions of differing composition. (c) A fi nite-element discretization is much better suited for such systems.

Material A

Material B

Material C

(a) (b) (c)

cha9792x_ch31_890-914.indd Page 890 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 890 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.1 THE GENERAL APPROACH 891

the PDE can be developed for each of these elements. The total solution is then generated by linking together, or “assembling,” the individual solutions taking care to ensure continu- ity at the interelement boundaries. Thus, the PDE is satisfi ed in a piecewise fashion. As in Fig. 31.1c, the use of elements, rather than a rectangular grid, provides a much better approximation for irregularly shaped systems. Further, values of the unknown can be generated continuously across the entire solution domain rather than at isolated points. Because a comprehensive description is beyond the scope of this book, this chapter provides a general introduction to the fi nite-element method. Our primary objective is to make you comfortable with the approach and cognizant of its capabilities. In this spirit, the following section is devoted to a general overview of the steps involved in a typical fi nite-element solution of a problem. This is followed by a simple example: a steady-state, one-dimensional heated rod. Although this example does not involve PDEs, it allows us to develop and demonstrate major aspects of the fi nite-element approach unencumbered by complicating factors. We can then discuss some issues involved in employing the fi nite- element method for PDEs.

31.1 THE GENERAL APPROACH

Although the particulars will vary, the implementation of the fi nite-element approach usually follows a standard step-by-step procedure. The following provides a brief over- view of each of these steps. The application of these steps to engineering problem con- texts will be developed in subsequent sections.

31.1.1 Discretization

This step involves dividing the solution domain into fi nite elements. Figure 31.2 provides examples of elements employed in one, two, and three dimensions. The points of inter- section of the lines that make up the sides of the elements are referred to as nodes and the sides themselves are called nodal lines or planes.

31.1.2 Element Equations

The next step is to develop equations to approximate the solution for each element. This involves two steps. First, we must choose an appropriate function with unknown coef- fi cients that will be used to approximate the solution. Second, we evaluate the coeffi cients so that the function approximates the solution in an optimal fashion.

Choice of Approximation Functions. Because they are easy to manipulate mathemat- ically, polynomials are often employed for this purpose. For the one-dimensional case, the simplest alternative is a fi rst-order polynomial or straight line,

u(x) 5 a0 1 a1x (31.1)

where u(x) 5 the dependent variable, a0 and a1 5 constants, and x 5 the independent variable. This function must pass through the values of u(x) at the end points of the element at x1 and x2. Therefore,

u1 5 a0 1 a1x1 u2 5 a0 1 a1x2

cha9792x_ch31_890-914.indd Page 891 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 891 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

892 FINITE-ELEMENT METHOD

where u1 5 u(x1) and u2 5 u(x2). These equations can be solved using Cramer’s rule for

a0 5 u1 x2 2 u2 x1

x2 2 x1   a1 5

u2 2 u1 x2 2 x1

These results can then be substituted into Eq. (31.1) which, after collection of terms, can be written as

u 5 N1 u1 1 N2 u2 (31.2)

where

N1 5 x2 2 x x2 2 x1

(31.3)

and

N2 5 x 2 x1 x2 2 x1

(31.4)

Equation (31.2) is called an approximation, or shape, function, and N1 and N2 are called interpolation functions. Close inspection reveals that Eq. (31.2) is, in fact, the Lagrange

FIGURE 31.2 Examples of elements employed in (a) one, (b) two, and (c) three dimensions.

Line element

(a) One-dimensional

Nodal line

Node

Triangular element

Quadrilateral element

(b) Two-dimensional

Hexahedron element

Nodal plane

(c) Three-dimensional

cha9792x_ch31_890-914.indd Page 892 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 892 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.1 THE GENERAL APPROACH 893

fi rst-order interpolating polynomial. It provides a means to predict intermediate values (that is, to interpolate) between given values u1 and u2 at the nodes. Figure 31.3 shows the shape function along with the corresponding interpolation functions. Notice that the sum of the interpolation functions is equal to one. In addition, the fact that we are dealing with linear equations facilitates operations such as differentiation and integration. Such manipulations will be important in later sections. The derivative of Eq. (31.2) is

du

dx 5

dN1 dx

u1 1 dN2 dx

u2 (31.5)

According to Eqs. (31.3) and (31.4), the derivatives of the N’s can be calculated as

dN1 dx

5 2 1

x2 2 x1   

dN2 dx

5 1

x2 2 x1 (31.6)

and, therefore, the derivative of u is

du

dx 5

1 x2 2 x1

(2u1 1 u2) (31.7)

In other words, it is a divided difference representing the slope of the straight line con- necting the nodes. The integral can be expressed as

# x2

x1

u dx 5 # x2

x1

N1 u1 1 N2 u2 dx

Each term on the right-hand side is merely the integral of a right triangle with base x2 2 x1 and height u. That is,

# x2

x1 Nu dx 5

1

2 (x2 2 x1)u

Thus, the entire integral is

# x2

x1

u dx 5 u1 1 u2

2 (x2 2 x1) (31.8)

In other words, it is simply the trapezoidal rule.

Obtaining an Optimal Fit of the Function to the Solution. Once the interpolation function is chosen, the equation governing the behavior of the element must be devel- oped. This equation represents a fi t of the function to the solution of the underlying differential equation. Several methods are available for this purpose. Among the most common are the direct approach, the method of weighted residuals, and the variational approach. The outcome of all of these methods is analogous to curve fi tting. However, instead of fi tting functions to data, these methods specify relationships between the un- knowns in Eq. (31.2) that satisfy the underlying PDE in an optimal fashion.

FIGURE 31.3 (b) A linear approximation or shape function for (a) a line element. The corresponding interpolation functions are shown in (c ) and (d ).

Node 1 Node 2

u1

u2

x1 x2

N1

N2

u

1

1

(a)

(b)

(c)

(d)

cha9792x_ch31_890-914.indd Page 893 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 893 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

894 FINITE-ELEMENT METHOD

Mathematically, the resulting element equations will often consist of a set of linear algebraic equations that can be expressed in matrix form,

[k]{u} 5 {F} (31.9)

where [k] 5 an element property or stiffness matrix, {u} 5 a column vector of un- knowns at the nodes, and {F} 5 a column vector refl ecting the effect of any external infl uences applied at the nodes. Note that, in some cases, the equations can be nonlin- ear. However, for the elementary examples described herein, and for many practical problems, the systems are linear.

31.1.3 Assembly

After the individual element equations are derived, they must be linked together or as- sembled to characterize the unifi ed behavior of the entire system. The assembly process is governed by the concept of continuity. That is, the solutions for contiguous elements are matched so that the unknown values (and sometimes the derivatives) at their common nodes are equivalent. Thus, the total solution will be continuous. When all the individual versions of Eq. (31.9) are fi nally assembled, the entire sys- tem is expressed in matrix form as

[k]{u¿} 5 {F¿} (31.10)

where [K] 5 the assemblage property matrix and {u9} and {F9} column vectors for un- knowns and external forces that are marked with primes to denote that they are an as- semblage of the vectors {u} and {F} from the individual elements.

31.1.4 Boundary Conditions

Before Eq. (31.10) can be solved, it must be modifi ed to account for the system’s bound- ary conditions. These adjustments result in

[k]{u¿} 5 {F ¿} (31.11)

where the overbars signify that the boundary conditions have been incorporated.

31.1.5 Solution

Solutions of Eq. (31.11) can be obtained with techniques described previously in Part Three, such as LU decomposition. In many cases, the elements can be confi gured so that the resulting equations are banded. Thus, the highly effi cient solution schemes available for such systems can be employed.

31.1.6 Postprocessing

Upon obtaining a solution, it can be displayed in tabular form or graphically. In addition, secondary variables can be determined and displayed. Although the preceding steps are very general, they are common to most imple- mentations of the fi nite-element approach. In the following section, we illustrate how they can be applied to obtain numerical results for a simple physical system—a heated rod.

cha9792x_ch31_890-914.indd Page 894 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 894 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION 895

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION

Figure 31.4 shows a system that can be modeled by a one-dimensional form of Poisson’s equation

d2 T

dx2 5 2f (x) (31.12)

where f(x) 5 a function defi ning a heat source along the rod and where the ends of the rod are held at fi xed temperatures,

T(0, t) 5 T1

and

T(L, t) 5 T2

Notice that this is not a partial differential equation but rather is a boundary-value ODE. This simple model is used because it will allow us to introduce the fi nite-element approach without some of the complications involved in, for example, a two-dimensional PDE.

EXAMPLE 31.1 Analytical Solution for a Heated Rod

Problem Statement. Solve Eq. (31.12) for a 10-cm rod with boundary conditions of T(0, t) 5 40 and T(10, t) 5 200 and a uniform heat source of f(x) 5 10.

Solution. The equation to be solved is

d 2 T

dx2 5 210

1 2 3 4

1 2 3 4 5

x = 0 x = L

T(0, t) T(L, t)

f (x)

x

(a)

(b)

FIGURE 31.4 (a) A long, thin rod subject to fi xed boundary conditions and a continuous heat source along its axis. (b) The fi nite-element representation consisting of four equal-length elements and fi ve nodes.

cha9792x_ch31_890-914.indd Page 895 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 895 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

896 FINITE-ELEMENT METHOD

Assume a solution of the form

T 5 ax2 1 bx 1 c

which can be differentiated twice to give T 0 5 2a. Substituting this result into the dif- ferential equation gives a 5 25. The boundary conditions can be used to evaluate the remaining coeffi cients. For the fi rst condition at x 5 0,

40 5 25(0)2 1 b(0) 1 c

or c 5 40. Similarly, for the second condition,

200 5 25(10)2 1 b(10) 1 40

which can be solved for b 5 66. Therefore, the fi nal solution is

T 5 25x2 1 66x 1 40

The results are plotted in Fig. 31.5.

FIGURE 31.5 The temperature distribution along a heated rod subject to a uniform heat source and held at fi xed end temperatures.

T

100

0

200

5 10 x

31.2.1 Discretization

A simple confi guration to model the system is a series of equal-length elements (Fig. 31.4b). Thus, the system is treated as four equal-length elements and fi ve nodes.

cha9792x_ch31_890-914.indd Page 896 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 896 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION 897

31.2.2 Element Equations

An individual element is shown in Fig. 31.6a. The distribution of temperature for the element can be represented by the approximation function

T̃ 5 N1T1 1 N2T2 (31.13)

where N1 and N2 5 linear interpolation functions specifi ed by Eqs. (31.3) and (31.4), respectively. Thus, as depicted in Fig. 31.6b, the approximation function amounts to a linear interpolation between the two nodal temperatures. As noted in Sec. 31.1, there are a variety of approaches for developing the element equation. In this section, we employ two of these. First, a direct approach will be used for the simple case where f(x) 5 0. Then, because of its general applicability in engi- neering, we will devote most of the section to the method of weighted residuals.

The Direct Approach. For the case where f(x) 5 0, a direct method can be employed to generate the element equations. The relationship between heat fl ux and temperature gradient can be represented by Fourier’s law:

q 5 2k¿ d T

dx

where q 5 flux [cal/(cm2 ? s)] and k9 5 the coefficient of thermal conductivity [cal/(s ? cm ? 8C)]. If a linear approximation function is used to characterize the element’s temperature, the heat fl ow into the element through node 1 can be represented by

q1 5 k¿ T1 2 T2 x2 2 x1

where q1 is heat fl ux at node 1. Similarly, for node 2,

q2 5 k¿ T2 2 T1 x2 2 x1

These two equations express the relationship of the element’s internal temperature dis- tribution (as refl ected by the nodal temperatures) to the heat fl ux at its ends. As such, they constitute our desired element equations. They can be simplifi ed further by recog- nizing that Fourier’s law can be used to couch the end fl uxes themselves in terms of the temperature gradients at the boundaries. That is,

q1 5 2k¿ dT(x1)

dx   q2 5 k¿

dT(x2)

dx

which can be substituted into the element equations to give

1 x2 2 x1

c 1 21 21 1

d eT1 T2 f 5 μ

2 dT(x1)

dx dT(x2)

dx

∂ (31.14)

Notice that Eq. (31.14) has been cast in the format of Eq. (31.9). Thus, we have succeeded in generating a matrix equation that describes the behavior of a typical element in our system.

FIGURE 31.6 (a) An individual element. (b) The approximation function used to characterize the temperature distribution along the element.

Node 1 Node 2

T1

T2

x1 x2

T

(a)

(b)

~

cha9792x_ch31_890-914.indd Page 897 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 897 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

898 FINITE-ELEMENT METHOD

The direct approach has great intuitive appeal. Additionally, in areas such as mechan- ics, it can be employed to solve meaningful problems. However, in other contexts, it is often diffi cult or impossible to derive fi nite-element equations directly. Consequently, as described next, more general mathematical techniques are available.

The Method of Weighted Residuals. The differential equation (31.12) can be reex- pressed as

0 5 d2T

dx2 1 f(x)

The approximate solution [Eq. (31.13)] can be substituted into this equation. Because Eq. (31.13) is not the exact solution, the left side of the resulting equation will not be zero but will equal a residual,

R 5 d2T̃

dx2 1 f(x) (31.15)

The method of weighted residuals (MWR) consists of fi nding a minimum for the residual according to the general formula

# D

RWi dD 5 0  i 5 1, 2, p , m (31.16)

where D 5 the solution domain and the Wi 5 linearly independent weighting functions. At this point, there are a variety of choices that could be made for the weighting function (Box 31.1). The most common approach for the fi nite-element method is to employ the interpolation functions Ni as the weighting functions. When these are substi- tuted into Eq. (31.16), the result is referred to as Galerkin’s method,

# D

RNi dD 5 0  i 5 1, 2, p , m

For our one-dimensional rod, Eq. (31.15) can be substituted into this formulation to give

# x2

x1

c d2T̃ dx2

1 f(x)d Ni dx  i 5 1, 2 which can be reexpressed as

# x2

x1

d2T̃

dx2 Ni(x) dx 5 2#

x2

x1

f (x)Ni(x) dx  i 5 1, 2 (31.17)

At this point, a number of mathematical manipulations will be applied to simplify and evaluate Eq. (31.17). Among the most important is the simplifi cation of the left-hand side using integration by parts. Recall from calculus that this operation can be expressed generally as

# b

a u dy 5 uyZ ba 2 #

b

a y du

cha9792x_ch31_890-914.indd Page 898 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 898 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION 899

If u and y are chosen properly, the new integral on the right-hand side will be easier to evaluate than the original one on the left-hand side. This can be done for the term on the left-hand side of Eq. (31.17) by choosing Ni(x) as u and (d

2Tydx2) dx as dy to yield

# x2

x1 Ni(x)

d2T̃

dx2 dx 5 Ni(x)

dT̃

dx ` x2 x1

2 # x2

x1

dT̃

dx dNi dx

dx  i 5 1, 2 (31.18)

Thus, we have taken the signifi cant step of lowering the highest-order term in the for- mulation from a second to a fi rst derivative. Next, we can evaluate the individual terms that we have created in Eq. (31.18). For i 5 1, the fi rst term on the right-hand side of Eq. (31.18) can be evaluated as

N1(x) dT̃

dx ` x2 x1

5 N1(x2) dT̃(x2)

dx 2 N1(x1)

dT̃(x1)

dx

However, recall from Fig. 31.3 that N1(x2) 5 0 and N1(x1) 5 1, and therefore,

N1(x) dT̃

dx ` x2 x1

5 2 dT̃(x1)

dx (31.19)

Box 31.1 Alternative Residual Schemes for the MWR

Several choices can be made for the weighting functions of Eq. (31.16). Each represents an alternative approach for the MWR. In the collocation approach, we choose as many locations as there are unknown coeffi cients. Then, the coeffi cients are adjusted until the residual vanishes at each of these locations. Consequently, the approximating function will yield perfect results at the chosen locations but will have a nonzero residual elsewhere. Thus, it is akin to the interpolation methods in Chap. 18. Note that collocation amounts to using the weighting function

W 5 d(x 2 xi)   for i 5 1, 2, p , n

where n 5 the number of unknown coeffi cients and d(x 2 xi) 5 the Dirac delta function that vanishes everywhere but at x 5 xi, where it equals 1. In the subdomain method, the interval is divided into as many segments, or “subdomains,” as there are unknown coeffi cients. Then, the coeffi cients are adjusted until the average value of the residual is zero in each subdomain. Thus, for each subdomain, the weighting function is equal to 1 and Eq. (31.16) is

# xi

xi21 R dx 5 0  for i 5 1, 2, p , n

where xi21 and xi are the bounds of the subdomain.

For the least-squares case, the coeffi cients are adjusted so as to minimize the integral of the square of the residual. Thus, the weighting functions are

Wi 5 0R 0ai

which can be substituted into Eq. (31.16) to give

# D

R 0R 0ai

dD 5 0  i 5 1, 2, p , n

or

0 0ai #D R

2 dD 5 0  i 5 1, 2, p , n

Comparison of the formulation with those of Chap. 17 shows that this is the continuous form of regression. Galerkin’s method employs the interpolation functions Ni as weighting functions. Recall that these functions always sum to 1 at any position in an element. For many problem contexts, Galerkin’s method yields the same results as are obtained by variational meth- ods. Consequently, it is the most commonly employed version of MWR used in fi nite-element analysis.

cha9792x_ch31_890-914.indd Page 899 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 899 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

900 FINITE-ELEMENT METHOD

Similarly, for i 5 2,

N2(x) dT̃

dx ` x2 x1

5 dT̃(x2)

dx (31.20)

Thus, the fi rst term on the right-hand side of Eq. (31.18) represents the natural boundary conditions at the ends of the elements. Now, before proceeding let us regroup by substituting our results back into the original equation. Substituting Eqs. (31.18) through (31.20) into Eq. (31.17) and rear- ranging gives for i 5 1,

# x2

x1

dT̃

dx dN1 dx

dx 5 2 dT̃(x1)

dx 1 #

x2

x1 f(x)N1(x) dx (31.21)

and for i 5 2,

# x2

x1

dT̃

dx dN2 dx

dx 5 dT̃(x2)

dx 1 #

x2

x1 f(x)N2(x) dx (31.22)

Notice that the integration by parts has led to two important outcomes. First, it has incorporated the boundary conditions directly into the element equations. Second, it has lowered the highest-order evaluation from a second to a fi rst derivative. This latter out- come yields the signifi cant result that the approximation functions need to preserve con- tinuity of value but not slope at the nodes. Also notice that we can now begin to ascribe some physical signifi cance to the in- dividual terms we have derived. On the right-hand side of each equation, the fi rst term represents one of the element’s boundary conditions and the second is the effect of the system’s forcing function—in the present case, the heat source f(x). As will now become evident, the left-hand side embodies the internal mechanisms that govern the element’s temperature distribution. That is, in terms of the fi nite-element method, the left-hand side will become the element property matrix. To see this, let us concentrate on the terms on the left-hand side. For i 5 1, the term is

# x2

x1

dT̃

dx dN1 dx

dx (31.23)

Recall from Sec. 31.1.2 that the linear nature of the shape function makes differentiation and integration simple. Substituting Eqs. (31.6) and (31.7) into Eq. (31.23) gives

# x2

x1

T1 2 T2

(x2 2 x1) 2 dx 5

1 x2 2 x1

(T1 2 T2) (31.24)

Similar substitutions for i 5 2 [Eq. (31.22)] yield

# x2

x1

2T1 1 T2 (x2 2 x1)

2 dx 5 1

x2 2 x1 (2T1 1 T2) (31.25)

Comparison with Eq. (31.14) shows that these are similar to the relationships that were developed with the direct method using Fourier’s law. This can be made even clearer

cha9792x_ch31_890-914.indd Page 900 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 900 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION 901

by reexpressing Eqs. (31.24) and (31.25) in matrix form as

1 x2 2 x1

c 1 21 21 1

d eT1 T2 f

Substituting this result into Eqs. (31.21) and (31.22) and expressing the result in matrix form gives the fi nal version of the element equations

1 x2 2 x1

c 1 21 21 1

d {T} 5 μ 2

dT(x1)

dx dT(x2)

ds

∂ 1 μ # x2

x1 f(x)N1(x) dx

# x2

x1 f(x)N2(x) dx

∂ (31.26)

Note that aside from the direct and the weighted residual methods, the element equa- tions can also be derived using variational calculus (for example, see Allaire, 1985). For the present case, this approach yields equations that are identical to those derived above.

EXAMPLE 31.2 Element Equation for a Heated Rod

Problem Statement. Employ Eq. (31.26) to develop the element equations for a 10-cm rod with boundary conditions of T(0, t) 5 40 and T(10, t) 5 200 and a uniform heat source of f(x) 5 10. Employ four equal-size elements of length 5 2.5 cm.

Solution. The heat source term in the fi rst row of Eq. (31.26) can be evaluated by substituting Eq. (31.3) and integrating to give

# 2.5

0 10

2.5 2 x

2.5 dx 5 12.5

Similarly, Eq. (31.4) can be substituted into the heat source term of the second row of Eq. (31.26), which can also be integrated to yield

# 2.5

0 10

x 2 0

2.5 dx 5 12.5

These results along with the other parameter values can be substituted into Eq. (31.26) to give

0.4T1 2 0.4T2 5 2 dT

dx (x1) 1 12.5

and

20.4T1 1 0.4T2 5 dT

dx (x2) 1 12.5

31.2.3 Assembly

Before the element equations are assembled, a global numbering scheme must be estab- lished to specify the system’s topology or spatial layout. As in Table 31.1, this defi nes the

External effects

sf fElement stiffness matrix Boundary condition

cha9792x_ch31_890-914.indd Page 901 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 901 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

902 FINITE-ELEMENT METHOD

TABLE 31.1 The system topology for the fi nite-element segmentation scheme from Fig. 31.4b.

Node Numbers

Element Local Global

1 1 1 2 2 2 1 2 2 3 3 1 3 2 4 4 1 4 2 5

FIGURE 31.7 The assembly of the equations for the total system.

(a)

0.4 20.4 0 0 0 20.4 0.4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

  

T1 T2 0 0 0

  5 

2dT(x1)ydx 1 12.5 dT(x2)ydx 1 12.5

0 0 0

(b)

0.4 20.4 0 0 0

20.4 0.4

0 0

10.4 20.4

0 20.4

0.4 0 0

0 0 0 0 0

0 0 0 0 0

T1 T2 T3 0 0

  5 

2dT(x1)ydx 1 12.5 12.5 1 12.5

dT(x3)ydx 1 12.5 0 0

(c)

0.4 20.4 0 0 0

20.4 0.8

20.4 0 0

0 20.4

0.4

0

10.4 20.4

0 0

20.4 0.4

0

0 0 0 0 0

T1 T2 T3 T4 0

  5 

2dT(x1)ydx 1 12.5 25

12.5 1 12.5 dT(x4)ydx 1 12.5

0

(d)

0.4 20.4 0 0 0

20.4 0.8

20.4 0 0

0 20.4

0.8 20.4 0

0 0

20.4 0.4

10.4 20.4

0 0 0

20.4 0.4

T1 T2 T3 T4 T5

  5 

2dT(x1)ydx 1 12.5 25 25

12.5 1 12.5 dT(x5)ydx 1 12.5

(e)

0.4 20.4 0 0 0

20.4 0.8

20.4 0 0

0 20.4

0.8 20.4 0

0 0

20.4 0.8

20.4

0 0 0

20.4 0.4

T1 T2 T3 T4 T5

  5 

2dT(x1)ydx 1 12.5 25 25 25

dT(x5)ydx 1 12.5

E

E

E

E E

E

E

E

E E

e

e

e

e e

e

e

e

e e

e

e

e

e e

e

e

e

e e

cha9792x_ch31_890-914.indd Page 902 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 902 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.2 FINITE-ELEMENT APPLICATION IN ONE DIMENSION 903

connectivity of the element mesh. Because the present case is one-dimensional, the numbering scheme might seem so predictable that it is trivial. However, for two- and three-dimensional problems it offers the only means to specify which nodes belong to which elements. Once the topology is specifi ed, the element equation (31.26) can be written for each element using the global coordinates. Then they can be added one at a time to assemble the total system matrix (note that this process is explored further in Sec. 32.4). The process is depicted in Fig. 31.7.

31.2.4 Boundary Conditions

Notice that, as the equations are assembled, the internal boundary conditions cancel. Thus, the fi nal result for {F} in Fig. 31.7e has boundary conditions for only the fi rst and the last nodes. Because T1 and T5 are given, these natural boundary conditions at the ends of the bar, dT(x1)ydx and dT(x5)ydx, represent unknowns. Therefore, the equations can be re-expressed as

dT

dx (x1) 20.4T2 5 23.5

0.8T2 20.4T3 5    41

20.4T2 10.8T3 20.4T4 5    25

20.4T3 10.8T4 5 105

20.4T4 2 dT

dx (x5) 5 267.5

(31.27)

FIGURE 31.8 Results of applying the fi nite-element approach to a heated bar. The exact solution is also shown.

T

100

0

200

5 10 x

Finite-element

Analytical

cha9792x_ch31_890-914.indd Page 903 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 903 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

904 FINITE-ELEMENT METHOD

31.2.5 Solution

Equation (31.27) can be solved for

dT

dx (x1) 5 66  T2 5 173.75  T3 5 245

T4 5 253.75   dT

dx (x5) 5 234

31.2.6 Postprocessing

The results can be displayed graphically. Figure 31.8 shows the fi nite-element results along with the exact solution. Notice that the fi nite-element calculation captures the overall trend of the exact solution and, in fact, provides an exact match at the nodes. However, a discrepancy exists in the interior of each element due to the linear nature of the shape functions.

31.3 TWO-DIMENSIONAL PROBLEMS

Although the mathematical “bookkeeping” increases markedly, the extension of the fi nite- element approach to two dimensions is conceptually similar to the one-dimensional applica- tions discussed to this point. It thus follows the same steps as were outlined in Sec. 31.1.

31.3.1 Discretization

A variety of simple elements such as triangles or quadrilaterals are usually employed for the fi nite-element mesh in two dimensions. In the present discussion, we will limit our- selves to triangular elements of the type depicted in Fig. 31.9.

31.3.2 Element Equations

Just as for the one-dimensional case, the next step is to develop an equation to ap- proximate the solution for the element. For a triangular element, the simplest approach is the linear polynomial [compare with Eq. (31.1)]

u(x, y) 5 a0 1 a1,1x 1 a1,2y (31.28)

FIGURE 31.9 A triangular element.

y

x

3

2

1

cha9792x_ch31_890-914.indd Page 904 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 904 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.3 TWO-DIMENSIONAL PROBLEMS 905

where u(x, y) 5 the dependent variable, the a’s 5 coeffi cients, and x and y 5 independent variables. This function must pass through the values of u(x, y) at the triangle’s nodes (x1, y1), (x2, y2), and (x3, y3). Therefore,

u1(x, y) 5 a0 1 a1,1x1 1 a1, 2y1 u2(x, y) 5 a0 1 a1,1x2 1 a1, 2y2 u3(x, y) 5 a0 1 a1,1x3 1 a1, 2y3

or in matrix form,

£ 1 x1 y1 1 x2 y2 1 x3 y3

§ • a0 a1,1 a1, 2

¶ 5 • u1 u2 u3

which can be solved for

a0 5 1

2Ae [u1(x2y3 2 x3y2) 1 u2(x3y1 2 x1y3) 1 u3(x1y2 2 x2y1) ] (31.29)

a1,1 5 1

2Ae [u1(y2 2 y3) 1 u2(y3 2 y1) 1 u3(y1 2 y2) ] (31.30)

a1, 2 5 1

2Ae [u1(x3 2 x2) 1 u2(x1 2 x3) 1 u3(x2 2 x1) ] (31.31)

where Ae is the area of the triangular element,

Ae 5 1

2 [ (x2y3 2 x3y2) 1 (x3y1 2 x1y3) 1 (x1y2 2 x2y1) ]

Equations (31.29) through (31.31) can be substituted into Eq. (31.28). After a col- lection of terms, the result can be expressed as

u 5 N1u1 1 N2u2 1 N3u3 (31.32)

where

N1 5 1

2Ae [ (x2y3 2 x3y2) 1 (y2 2 y3)x 1 (x3 2 x2)y]

N2 5 1

2Ae [ (x3y1 2 x1y3) 1 (y3 2 y1)x 1 (x1 2 x3)y]

N3 5 1

2Ae [ (x1y2 2 x2y1) 1 (y1 2 y2)x 1 (x2 2 x1)y]

Equation (31.32) provides a means to predict intermediate values for the element on the basis of the values at its nodes. Figure 31.10 shows the shape function along with the corresponding interpolation functions. Notice that the sum of the interpolation func- tions is always equal to 1. As with the one-dimensional case, various methods are available for developing ele- ment equations based on the underlying PDE and the approximating functions. The result- ing equations are considerably more complicated than Eq. (31.26). However, because the

cha9792x_ch31_890-914.indd Page 905 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 905 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

906 FINITE-ELEMENT METHOD

approximating functions are usually lower-order polynomials like Eq. (31.28), the terms of the fi nal element matrix will consist of lower-order polynomials and constants.

31.3.3 Boundary Conditions and Assembly

The incorporation of boundary conditions and the assembly of the system matrix also become more complicated when the fi nite-element technique is applied to two- and three- dimensional problems. However, as with the derivation of the element matrix, the diffi culty

FIGURE 31.10 (a) A linear approximation function for a triangular element. The corresponding interpolation func- tions are shown in (b) through (d ).

u

x

y

u3 u2

u1

x

y

N1

1

0 0

x

y

N2

1 0

0

x

y

N3

1

0

0

(a)

(b)

(c)

(d)

cha9792x_ch31_890-914.indd Page 906 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 906 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.3 TWO-DIMENSIONAL PROBLEMS 907

FIGURE 31.11 A numbering scheme for the nodes and elements of a fi nite-element approximation of the heated plate that was previously characterized by fi nite differences in Chap. 29.

26

25

28

27

30

29

32

31

18

17

20

19

22

21

24

23

10

9

12

11

14

13

16

15

2

1

4

3

6

5

8

7

1 2 3 4 5

10

15

20

2524232221

16 17 18 19

11 12 13 14

6 7 8 9

100 100 100 100 100

50

50

50

00000

75

75

75

FIGURE 31.12 The temperature distribution of a heated plate as calculated with a fi nite-element method.

cha9792x_ch31_890-914.indd Page 907 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 907 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

908 FINITE-ELEMENT METHOD S O F T W A R E

relates to the mechanics of the process rather than to conceptual complexity. For example, the establishment of the system topology, which was trivial for the one-dimensional case, becomes a matter of great importance in two and three dimensions. In particular, the choice of a numbering scheme will dictate the bandedness of the resulting system matrix and hence the effi ciency with which it can be solved. Figure 31.11 shows a scheme that was developed for the heated plate formerly solved by fi nite-difference methods in Chap. 29.

31.3.4 Solution and Postprocessing

Although the mechanics are complicated, the system matrix is merely a set of n simul- taneous equations that can be used to solve for the values of the dependent variable at the n nodes. Figure 31.12 shows a solution that corresponds to the fi nite-difference solu- tion from Fig. 29.5.

31.4 SOLVING PDES WITH SOFTWARE PACKAGES

Software packages have some capabilities for directly solving PDEs. However, as de- scribed in the following sections, many of the solutions are limited to simple problems. This is particularly true of two- and three-dimensional cases. For these situations, generic packages (that is, ones not expressly developed to solve PDEs such as fi nite-element packages) are often limited to simple rectangular domains. Although this might seem limiting, simple applications can be of great utility in a pedagogical sense. This is particularly true when the packages’ visualization tools are used to display calculation results.

31.4.1 Excel

Although Excel does not have the direct capability to solve PDEs, it is a nice environ- ment to develop simple solutions of elliptic PDEs. For example, the orthogonal layout of the spreadsheet cells (Fig. 31.13b) is directly analogous to the grid used in Chap. 29 to model the heated plate (Fig. 31.13a).

FIGURE 31.13 The analogy between (a) a rect- angular grid and (b) the cells of a spreadsheet.

78.57 100 B

42.86 0

63.17 75

87.5 A

75 37.5

75 69.64 100 D

33.93 0

52.46 50 75 E

50 25

50 76.12 100 C

33.26 0

56.25

1 2 3 4 5

87.5 100 100

75

75

T13 = 100 + T23 + T12 + 75

4 4 B2 = B1 + C2 + B3 + A2

(a) Grid (b) Spreadsheet

cha9792x_ch31_890-914.indd Page 908 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 908 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.4 SOLVING PDES WITH SOFTWARE PACKAGES 909

As in Fig. 31.13b, the Dirichlet boundary conditions can fi rst be entered along the periphery of the cell block. The formula for the Liebmann method can be implemented by entering Eq. (29.11) in one of the cells in the interior (like cell B2 in Fig. 31.13b). Thus, the value for the cell can be computed as a function of its adjacent cells. Then the cell can be copied to the other interior cells. Because of the relative nature of the Excel copy command, all the other cells will properly be dependent on their adjacent cells. Once you have copied the formula, you will probably get an error message: Cannot resolve circular references. You can rectify this by selecting File, Options and clicking on the Formulas category. Then, go to the Calculation options section and enable the Iterative calculation check box. This will allow the spreadsheet to recalculate (the default is 100 iterations) and solve Liebmann’s method iteratively. After this occurs, strike the F9 key to manually recalculate the sheet until the answers do not vary. This means that the solution has converged. Once the problem has been solved, Excel’s graphics tools can be used to visualize the results. An example is shown in Fig. 31.14a. For this case, we have

Used a finer grid. Made the lower boundary insulated. Added a heat source of 150 to the middle of the plate (cell E5).

FIGURE 31.14 (a) Excel solution of the Poisson equation for a plate with an insulated lower edge and a heat source. (b) A “topographic map” and (c) a 3-D display of the temperatures.

89.2 100.0

B

85.7 85.5

86.2 75.0 87.5 A

75.0 75.0

75.0 99.1

100.0 D

106.7 114.3

100.9 99.7

100.0 E

115.3 150.0

103.1 95.8

100.0 C

96.1 97.4

94.7 96.6

100.0 F

101.4 108.6

96.7 77.6

100.0 H

68.2 67.3

70.3 50.0 75.0

I

50.0 50.0

50.0 89.9

100.0 G

85.2 85.6

85.5

1 2 3 4 5

84.0

80.9 80.4

82.2 75.0

75.0 75.0

75.0 103.4

88.9 87.3

94.2 111.6

88.4 86.3

95.6 93.4

85.9 84.9

88.9 97.4

82.8 81.1

88.1 65.6

62.2 61.7

63.6 50.0

50.0 50.0

50.0 81.3

73.5 72.4

76.6 6 7 8 9

(b)

(a)

(c)

S9

S8

S7

S6

S5

S4

S3

S2

S1 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9

160

140

120

100

80

60

40

S1

S3

S5 S7

S9

cha9792x_ch31_890-914.indd Page 909 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 909 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

910 FINITE-ELEMENT METHOD

The numerical results from Fig. 31.14a can then be displayed with Excel’s Chart Wizard. Figure 31.14b and c show 3-D surface plots. The y orientation of these are normally the reverse of the spreadsheet. Thus, the top high-temperature edge (100) would normally be displayed at the bottom of the plot. We reversed the y values on our sheet prior to plotting so that the graphs would be consistent with the spreadsheet. Notice how the graphs help you visualize what is going on. Heat fl ows down from the source toward the boundaries, forming a mountainlike shape. Heat also fl ows from the high-temperature boundary down to the two side edges. Notice how the heat fl ows preferentially toward the lower-temperature edge (50). Finally, notice how the tempera- ture gradient in the y dimension goes to zero at the lower insulated edge (0Ty0yS 0).

31.4.2 MATLAB

Although the standard MATLAB software package does not presently have great capabilities for solving PDEs, M-fi les and functions can certainly be developed for this purpose. In addition, its display capabilities are very nice, particularly for visualization of 2-D spatial problems. To illustrate this capability, we fi rst set up the Excel spreadsheet in Fig. 31.14a. These results can be saved as a text (Tab delimited) fi le with a name like plate.txt. This fi le can then be moved to the MATLAB directory. Once in MATLAB, the fi le can be loaded by typing

>> load plate.txt

Next, the gradients can be simply calculated as

>> [px,py]=gradient(plate);

Note that this is the simplest method to compute gradients using default values of dx 5 dy 5 1. Therefore, the directions and relative magnitudes will be correct. Finally, a series of commands can be used to develop the plot. The command contour develops a contour plot of the data. The command clabel adds contour labels to the plot. Finally, quiver takes the gradient data and adds it to the plot as arrows,

>> cs=contour(plate);clabel(cs);hold on

>> quiver(−px,−py);hold off

Note that the minus signs are added because of the minus sign in Fourier’s law [Eq. (29.4)]. As seen in Fig. 31.15, the resulting plot provides an excellent representation of the solution. Note that any fi le in the proper format can be entered into MATLAB and displayed in this way. This sharing of fi les between tools is becoming commonplace. In addition, fi les can be created in one location on one tool, transmitted over the Internet to another location, where the fi le might be displayed with another tool. This is one of the exciting aspects of modern numerical applications.

S O F T W A R E

cha9792x_ch31_890-914.indd Page 910 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 910 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

31.4 SOLVING PDES WITH SOFTWARE PACKAGES 911

FIGURE 31.15 MATLAB-generated contour plots for the heated plate calculated with Excel (Fig. 31.14).

++

+

+

+ +

+

+

+

++

9

8

7

6

5

4

3

2

1 1 2 3 4 5 6 7 8 9

6090

70

80

100

140 110

120 80

100

31.4.3 Mathcad

Mathcad has two functions that can solve Poisson’s equation. You can use the relax function when you know the value of the unknown on all four sides of a square region. This function solves a system of linear algebraic equations using Gauss-Seidel iteration with overrelaxation to speed the rate of convergence. For the special case where there are internal sources or sinks, and the unknown function is zero on all four sides of the square, then you can use the multigrid function, which is usually faster than relax. Both of these functions return a square matrix where the location of the element in the matrix corresponds to its location within the square region. The value of the element approxi- mates the value of the solution of Poisson’s equation at this point. Figure 31.16 shows an example where a square plate contains heat sources while the boundary is maintained at zero. The fi rst step is to establish dimensions for the temperature grid and the heat source matrix. The temperature grid has dimensions (R 1 1) 3 (R 1 1) while the heat source matrix is R 3 R. For example, a 3 3 3 temperature grid has 4 (2 3 2) possible heat sources. In this case, we establish a 33 3 33 temperature grid and a 32 3 32 heat source matrix. The Mathcad command MRR:5 0 (with R 5 32) establishes the dimensions of the source matrix and sets all the elements to zero. Next, the location and strength of two heat sources are established. Finally, S is the resulting temperature

cha9792x_ch31_890-914.indd Page 911 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 911 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

912 FINITE-ELEMENT METHOD

distribution as calculated by the multigrid function. The second argument of multigrid is a parameter that controls the numerical accuracy. As suggested by Mathcad help, a value of 2 generally gives a good approximation of the solution. The temperature distribution can be displayed with surface, contour, or vector-fi eld plots. These plots can be placed anywhere on the worksheet by clicking to the desired location. This places a red crosshair at that location. Then, use the Insert/Graph pull-down menu to place an empty plot on the worksheet with placeholders for the expressions to be graphed and for the ranges of variables. Simply type S in the placeholder on the z axis. Mathcad does the rest to produce the graphs shown in Fig. 31.16. Once the graph has been created, you can use the Format/Surface Plot and Format/Contour Plot pull- down menus to change the color or add titles, labels, and other features.

FIGURE 31.16 Mathcad screen to determine the solution of an elliptic PDE.

PROBLEMS

31.1 Repeat Example 31.1, but for T(0, t) 5 75 and T(10, t) 5 150 and a uniform heat source of 15. 31.2 Repeat Example 31.2, but for boundary conditions of T(0, t) 5 75 and T(10, t) 5 150 and a heat source of 15. 31.3 Apply the results of Prob. 31.2 to compute the temperature distribution for the entire rod using the fi nite-element approach. 31.4 Use Galerkin’s method to develop an element equation for a steady-state version of the advection-diffusion equation described

in Prob. 30.7. Express the fi nal result in the format of Eq. (31.26) so that each term has a physical interpretation. 31.5 A version of the Poisson equation that occurs in mechanics is the following model for the vertical defl ection of a bar with a dis- tributed load P(x):

AcE 02u 0x2

5 P(x)

S O F T W A R E

cha9792x_ch31_890-914.indd Page 912 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 912 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 913

where Ac 5 cross-sectional area, E 5 Young’s modulus, u 5 defl ec- tion, and x 5 distance measured along the bar’s length. If the bar is rigidly fi xed (u 5 0) at both ends, use the fi nite-element method to model its defl ections for Ac 5 0.1 m

2, E 5 200 3 109 N/m2, L 5 10 m, and P(x) 5 1000 N/m. Employ a value of Dx 5 2 m. 31.6 Develop a user-friendly program to model the steady-state distribution of temperature in a rod with a constant heat source us- ing the fi nite-element method. Set up the program so that unequally spaced nodes may be used. 31.7 Use Excel to perform the same computation as in Fig. 31.14, but insulate the right-hand edge and add a heat sink of 2150 at cell C7. 31.8 Use MATLAB or Mathcad to develop a contour plot with fl ux arrows for the Excel solution from Prob. 31.7. 31.9 Use Excel to model the temperature distribution of the slab shown in Fig. P31.9. The slab is 0.02 m thick and has a thermal conductivity of 3 W/(m ? 8C).

of the rod has a fi xed temperature gradient and the temperature is a variable. The right end has a fi xed temperature and the gradient is a variable. The heat source f(x) has a constant value. Thus, the condi- tions are

dT

0x ` x50

5 0.25°C/m T Z x550 5 100°C f (x) 5 30 W/cm

Develop the nodal equations that must be solved for the tempera- tures and temperature gradients at each of the six nodes. Assemble the equations, insert the boundary conditions, and solve the result- ing set for the unknowns. 31.12 Find the temperature distribution in a rod (Fig. P31.12) with internal heat generation using the fi nite-element method. Derive the element nodal equations using Fourier heat conduction.

qk 5 2kA dT

0x

and heat conservation relationships

a [qk 1 f (x) ] 5 0

100�C

50�C

75�C 25�C

2 m

0.6 m

1 m 0.4 m

–100 W/m2

FIGURE P31.9

31.10 Use MATLAB or Mathcad to develop a contour plot with fl ux arrows for the Excel solution from Prob. 31.9. 31.11 Find the temperature distribution in a rod (Fig. P31.11) with internal heat generation using the fi nite-element method. Derive the element nodal equations using Fourier heat conduction

qk 5 2kA dT

0x

and heat conservation relationships

a [qk 1 f (x) ] 5 0

where qk 5 heat fl ow (W), k 5 thermal conductivity (W/(m ? 8C)), A 5 cross-sectional area (m2), and f(x) 5 heat source (W/cm). The rod has a value of kA 5 100 W m/8C. The rod is 50 cm long, the x-coordinate is zero at the left end, and positive to the right. Divide the rod into fi ve elements (six nodes, each 10 cm long). The left end

FIGURE P31.11

kA = 100 W/m ·�C

f(x) = 30 W/cm

50 cm

dT–– dx x=0

= 0.25�C/m

T x=50 = 100�Cx

FIGURE P31.12

kA = 50 W·m/�C

kA = 100 W·m/�C f(x) = 30 W/cm

50 cm Tx=0 = 100�C

x

Tx=50 = 50�C

cha9792x_ch31_890-914.indd Page 913 06/11/13 8:22 PM F-468 cha9792x_ch31_890-914.indd Page 913 06/11/13 8:22 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

914 FINITE-ELEMENT METHOD

where qk 5 heat fl ow (W), k 5 thermal conductivity [W/(m ? 8C)], A 5 cross-sectional area (m2), and f(x) 5 heat source (W/cm). The rod is 50 cm long, the x-coordinate is zero at the left end, and posi- tive to the right. The rod is also linearly tapered with a value of kA 5 100 and 50 W m/8C at x 5 0 and at x 5 50, respectively. Divide the rod into fi ve elements (six nodes, each 10 cm long). Both ends of the rod have fi xed temperatures. The heat source f(x) has a constant value. Thus, the conditions are

T Z x50 5 100°C T Z x550 5 50°C f (x) 5 30 W/cm

The tapered areas must be treated as if they were constant over the length of an element. Therefore, average the kA values at each end of the node and take that average as a constant over the node. Develop the nodal equations that must be solved for the temperatures and temperature gradients at each of the six nodes. Assemble the equations, insert the boundary conditions, and solve the resulting set for the unknowns. 31.13 Use a software package to solve for the temperature distribu- tion of the L-shaped plate in Fig. P29.18. Display your results as a contour plot with fl ux arrows.

cha9792x_ch31_890-914.indd Page 914 06/11/13 8:23 PM F-468 cha9792x_ch31_890-914.indd Page 914 06/11/13 8:23 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

915

32 C H A P T E R 32

Case Studies: Partial Differential Equations

The purpose of this chapter is to apply the methods from Part Eight to practical engineer- ing problems. In Sec. 32.1, a parabolic PDE is used to compute the time-variable distri- bution of a chemical along the longitudinal axes of a rectangular reactor. This example illustrates how the instability of a solution can be due to the nature of the PDE rather than to properties of the numerical method. Sections 32.2 and 32.3 involve applications of the Poisson and Laplace equations to civil and electrical engineering problems, respectively. Among other things, this will allow you to see similarities as well as differences between fi eld problems in these areas of engineering. In addition, they can be contrasted with the heated-plate problem that has served as our prototype system in this part of the book. Section 32.2 deals with the defl ection of a square plate, whereas Sec. 32.3 is devoted to computing the voltage dis- tribution and charge fl ux for a two-dimensional surface with a curved edge. Section 32.4 presents a fi nite-element analysis as applied to a series of springs. This application is closer in spirit to fi nite-element applications in mechanics and structures than was the temperature fi eld problem used to illustrate the approach in Chap. 31.

32.1 ONE-DIMENSIONAL MASS BALANCE OF A REACTOR (CHEMICAL/BIO ENGINEERING)

Background. Chemical engineers make extensive use of idealized reactors in their design work. In Secs. 12.1 and 28.1, we focused on single or coupled well-mixed reactors. These are examples of lumped-parameter systems (recall Sec. PT3.1.2).

FIGURE 32.1 An elongated reactor with a single entry and exit point. A mass balance is developed around a fi nite segment along the tank’s longitudinal axis in order to derive a differential equation for the concentration. �x

x = 0 x = L

cha9792x_ch32_915-930.indd Page 915 05/11/13 6:55 PM F-468 cha9792x_ch32_915-930.indd Page 915 05/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

916 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

Figure 32.1 depicts an elongated reactor with a single entry and exit point. This reac- tor can be characterized as a distributed-parameter system. If it is assumed that the chem- ical being modeled is subject to fi rst-order decay1 and the tank is well-mixed vertically and laterally, a mass balance can be performed on a fi nite segment of length Dx, as in

V ¢c ¢t

5 Qc(x) Flow in

2 Q c c(x) 1 0c(x) 0x

¢x d Flow out

2 DAc 0c(x) 0x

Dispersion in

1DAc c 0c(x)0x 1 0 0x

0c(x) 0x

¢x d Dispersion out

2 kVc Decay reaction

(32.1)

where V 5 volume (m3), Q 5 fl ow rate (m3/h), c is concentration (moles/m3), D is a dispersion coeffi cient (m2/h), Ac is the tank’s cross-sectional area (m

2), and k is the fi rst- order decay coeffi cient (h21). Note that the dispersion terms are based on Fick’s fi rst law,

Flux 5 2D 0c 0x

(32.2)

which is directly analogous to Fourier’s law for heat conduction [recall Eq. (29.4)]. It specifi es that turbulent mixing tends to move mass from regions of high to low concen- tration. The parameter D, therefore, refl ects the magnitude of turbulent mixing. If Dx and Dt are allowed to approach zero, Eq. (32.1) becomes

0c 0t

5 D 02c 0x2

2 U 0c 0x

2 kc (32.3)

where U 5 QyAc is the velocity of the water fl owing through the tank. The mass balance for Fig. 32.1 is, therefore, now expressed as a parabolic partial differential equation. Equation (32.3) is sometimes referred to as the advection-dispersion equation with fi rst- order reaction. At steady state, it is reduced to a second-order ODE,

0 5 D d2c

dx2 2 U

d 2c

dx 2 kc (32.4)

Prior to t 5 0, the tank is fi lled with water that is devoid of the chemical. At t 5 0, the chemical is injected into the reactor’s infl ow at a constant level of cin. Thus, the fol- lowing boundary conditions hold:

Qcin 5 Qc0 2 DAc 0c0 0x

and

c¿(L, t) 5 0

The second condition specifi es that the chemical leaves the reactor purely as a function of fl ow through the outlet pipe. That is, it is assumed that dispersion in the reactor does

1That is, the chemical decays at a rate that is linearly proportional to how much chemical is present.

⎧ ⎨ ⎩

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ ⎧ ⎪ ⎨ ⎪ ⎩

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩

cha9792x_ch32_915-930.indd Page 916 05/11/13 6:55 PM F-468 cha9792x_ch32_915-930.indd Page 916 05/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.1 ONE-DIMENSIONAL MASS BALANCE OF A REACTOR 917

not affect the exit rate. Under these conditions, use numerical methods to solve Eq. (32.4) for the steady-state levels in the reactor. Note that this is an ODE boundary-value problem. Then solve Eq. (32.3) to characterize the transient response—that is, how the levels change in time as the system approaches the steady state. This application in- volves a PDE.

Solution. A steady-state solution can be developed by substituting centered fi nite dif- ferences for the fi rst and the second derivatives in Eq. (32.4) to give

0 5 D ci11 2 2ci 1 ci21

¢x2 2 U

ci11 2 ci21 2¢x

2 kci

Collecting terms gives

2a D U¢x

1 1

2 b ci21 1 a 2DU¢x 1

k ¢x U b c0 2 a DU¢x 2

1

2 b ci11 5 0 (32.5)

This equation can be written for each of the system’s nodes. At the reactor’s ends, this process introduces nodes that lie outside the system. For example, at the inlet node (i 5 0),

2a D U¢x

1 1

2 b c21 1 a 2DU¢x 1

k ¢x U b c0 2 a DU¢x 2

1

2 b c1 5 0 (32.6)

The c21 can be removed by invoking the fi rst boundary condition. At the inlet, the following mass balance must hold:

Qcin 5 Qc0 2 DAc 0c0 0x

where c0 5 concentration at x 5 0. Thus, this boundary condition specifi es that the amount of chemical carried into the tank by advection through the pipe must be equal to the amount carried away from the inlet by both advection and turbulent dispersion in the tank. A fi nite divided difference can be substituted for the derivative

Qcin 5 Qc0 2 DAc c1 2 c21

2 ¢x

which can be solved for

c21 5 c1 1 2 ¢xU

D cin 2

2 ¢xU D

c0

which can be substituted into Eq. (32.6) to give

a 2 D U ¢x

1 k ¢x

U 1 2 1

¢x U D b c0 2 a 2DU¢xb c1 5 a2 1

¢xU D b cin (32.7)

A similar exercise can be performed for the outlet, where the original difference equation is

2a D U¢x

1 1

2 b cn21 1 a 2 DU¢x 1

k ¢x U b cn 2 a DU¢x 2

1

2 b cn11 5 0 (32.8)

cha9792x_ch32_915-930.indd Page 917 05/11/13 6:55 PM F-468 cha9792x_ch32_915-930.indd Page 917 05/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

918 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

The boundary condition at the outlet is

Q cn 2 DAc dcn dx

5 Qcn

As with the inlet, a divided difference can be used to approximate the derivative.

Qcn 2 DAc cn11 2 cn21

2 ¢x 5 Qcn (32.9)

Inspection of this equation leads us to conclude that cn11 5 cn21. In other words, the slope at the outlet must be zero for Eq. (32.9) to hold. Substituting this result into Eq. (32.8) and simplifying gives

2a 2D U¢x

b cn21 1 a 2DU¢x 1 k ¢x

U b cn 5 0 (32.10)

Equations (32.5), (32.7), and (32.10) now form a system of n tridiagonal equations with n unknowns. For example, if D 5 2, U 5 1, Dx 5 2.5, k 5 0.2, and cin 5 100, the system is

E 5.35 21.621.3 2.1 20.321.3 2.1 20.3 21.3 2.1 20.3

21.6 2.1

U ec0c1c2 c3 c4

u 5 e32500 0

0

u which can be solved for

c0 5 76.44  c1 5 52.47  c2 5 36.06

c3 5 25.05  c4 5 19.09

These results are plotted in Fig. 32.2. As expected, the concentration decreases due to the decay reaction as the chemical fl ows through the tank. In addition to the above computation, Fig. 32.2 shows another case with D 5 4. Notice how increasing the tur- bulent mixing tends to fl atten the curve.

FIGURE 32.2 Concentration versus distance along the longitudinal axis of a rectangular reactor for a chemical that decays with fi rst- order kinetics.

c

20

40

60

80

100

0 2.5 5 7.5 10 x

D = 2

D = 4

cha9792x_ch32_915-930.indd Page 918 05/11/13 6:55 PM F-468 cha9792x_ch32_915-930.indd Page 918 05/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.2 DEFLECTIONS OF A PLATE 919

In contrast, if dispersion is decreased, the curve would become steeper as mixing became less important relative to advection and decay. It should be noted that if dispersion is decreased too much, the computation will become subject to numerical errors. This type of error is referred to as static instability to contrast it with the dynamic instability due to too large a time step during a dynamic computation. The criterion to avoid this static instability is

¢x # 2D

U

Thus, the criterion becomes more stringent (lower Dx) for cases where advection domi- nates over dispersion. Aside from steady-state computations, numerical methods can be used to generate time- variable solutions of Eq. (32.3). Figure 32.3 shows results for D 5 2, U 5 1, Dx 5 2.5, k 5 0.2, and cin 5 100, where the concentration in the tank is 0 at time zero. As expected, the immediate impact is near the inlet. With time, the solution eventually approaches the steady-state level. It should be noted that in such dynamic calculations, the time step is constrained by a stability criterion expressed as (Chapra, 1997)

¢t # (¢x)2

2D 1 k(¢x)2

Thus, the reaction term acts to make the time step smaller.

32.2 DEFLECTIONS OF A PLATE (CIVIL/ENVIRONMENTAL ENGINEERING)

Background. A square plate with simply supported edges is subject to an areal load q (Fig. 32.4). The defl ection in the z dimension can be determined by solving the elliptic PDE (see Carnahan, Luther, and Wilkes, 1969)

04z 0x4

1 2 04z 0x20y2

1 04z 0y4

5 q

D (32.11)

FIGURE 32.3 Concentration versus distance at different times during the buildup of chemical in a reactor.

c

100

100 x

Steady state t = 0.4

t = 0.8 t = 1.6 t = 3.2t = 0.2

cha9792x_ch32_915-930.indd Page 919 05/11/13 6:55 PM F-468 cha9792x_ch32_915-930.indd Page 919 05/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

920 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

subject to the boundary conditions that, at the edges, the defl ection and slope normal to the boundary are zero. The parameter D is the fl exural rigidity,

D 5 E ¢z3

12(1 2 s2) (32.12)

where E 5 the modulus of elasticity, Dz 5 the plate’s thickness, and s 5 Poisson’s ratio. If a new variable is defi ned as

u 5 02z 0x2

1 02z 0y2

Eq. (32.11) can be reexpressed as

02u 0x2

1 02u 0y2

5 q

D (32.13)

Therefore, the problem reduces to successively solving two Poisson equations. First, Eq. (32.13) can be solved for u subject to the boundary condition that u 5 0 at the edges. Then, the results can be employed in conjunction with

02z 0x2

1 02z 0y2

5 u (32.14)

to solve for z subject to the condition that z 5 0 at the edges. Develop a computer program to determine the defl ections for a square plate subject to a constant areal load. Test the program for a plate with 2-m-long edges, q 5 33.6 kN/m2, s 5 0.3, Dz 5 1022 m, and E 5 2 3 1011 Pa. Employ Dx 5 Dy 5 0.5 m for your test run.

Solution. Finite-divided differences can be substituted into Eq. (32.13) to give

ui11, j 2 2ui, j 1 ui21, j

¢x2 1

ui, j11 2 2ui, j 1 ui, j21

¢y2 5

q

D (32.15)

Equation (32.12) can be used to compute D 5 1.832 3 104 N/m. This result, along with the other system parameters, can be substituted into Eq. (32.15) to give

ui11, j 1 ui21, j 1 ui, j11 1 ui, j21 2 4ui, j 5 0.458

FIGURE 32.4 A simply supported square plate subject to an areal load.

y

z

x

�z

cha9792x_ch32_915-930.indd Page 920 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 920 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.3 TWO-DIMENSIONAL ELECTROSTATIC FIELD PROBLEMS 921

This equation can be written for all the nodes with the boundaries set at u 5 0. The resulting equations are

I 24 1 1

1 24 1 1

1 24 1

1 24 1 1

1 1 24 1 1

1 1 24 1

1 24 1

1 1 24 1

1 1 24

Y i u1, 1 u2, 1 u3, 1 u1, 2 u2, 2 u3, 2 u1, 3 u2, 3 u3, 3

y 5 i 0.458

0.458

0.458

0.458

0.458

0.458

0.458

0.458

0.458

y which can be solved for

u1, 1 5 20.315  u1, 2 5 20.401  u1, 3 5 20.315

u2, 1 5 20.401  u2, 2 5 20.515  u2, 3 5 20.401

u3, 1 5 20.315  u3, 2 5 20.401  u3, 3 5 20.315

These results in turn can be substituted into Eq. (32.14), which can be written in fi nite- difference form and solved for

z1, 1 5 0.063  z1, 2 5 0.086  z1, 3 5 0.063

z2, 1 5 0.086  z2, 2 5 0.118  z2, 3 5 0.086

z3, 1 5 0.063  z3, 2 5 0.086  z3, 3 5 0.063

32.3 TWO-DIMENSIONAL ELECTROSTATIC FIELD PROBLEMS (ELECTRICAL ENGINEERING)

Background. Just as Fourier’s law and the heat balance can be employed to character- ize temperature distribution, analogous relationships are available to model fi eld prob- lems in other areas of engineering. For example, electrical engineers use a similar approach when modeling electrostatic fi elds. Under a number of simplifying assumptions, an analog of Fourier’s law can be represented in one-dimensional form as

D 5 2e dV

dx

where D is called the electric fl ux density vector, e 5 permittivity of the material, and V 5 electrostatic potential. Similarly, a Poisson equation for electrostatic fi elds can be represented in two dimen- sions as

02V 0x2

1 02V 0y2

5 2 ry

e (32.16)

where ry 5 volumetric charge density.

cha9792x_ch32_915-930.indd Page 921 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 921 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

922 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

Finally, for regions containing no free charge (that is ry 5 0), Eq. (32.16) reduces to a Laplace equation,

02V 0x2

1 02V 0y2

5 0 (32.17)

Employ numerical methods to solve Eq. (32.17) for the situation depicted in Fig. 32.5. Compute both the values for V and for D if e 5 2.

Solution. Using the approach outlined in Sec. 29.3.2, Eq. (29.24) can be written for node (1, 1) as

2

¢x2 c V1,1 2 V0,1 a1(a1 1 a2)

1 V1,1 2 V2,1 a2(a1 1 a2)

d 1 2 ¢y2

c V1,1 2 V0,1 b1(b1 1 b2)

1 V1,1 2 V2,1 b2(b1 1 b2)

d 5 0 According to the geometry depicted in Fig. 32.5, Dx 5 3, Dy 5 2, b1 5 b2 5 a2 5 1, and a1 5 0.94281. Substituting these values yields

0.12132 V1, 1 2 121.32 1 0.11438 V1, 1 2 0.11438 V2, 1 1 0.25 V1, 1 10.25 V1, 1 2 0.25 V1, 2 5 0

Collecting terms gives

0.73570 V1,1 2 0.11438 V2,1 2 0.25 V1, 2 5 121.32

FIGURE 32.5 (a) A two-dimensional system with a voltage of 1000 along the circular boundary and a voltage of 0 along the base. (b) The nodal numbering scheme.

(a)

(b)

6

1000

1000

3

2

0 0

2,3 1,3

0,2

0,1

3,3

4,2

4,1

1,0 2,0 3,0

1,1 2,1 3,1

3,22,21,2

cha9792x_ch32_915-930.indd Page 922 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 922 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.3 TWO-DIMENSIONAL ELECTROSTATIC FIELD PROBLEMS 923

A similar approach can be applied to the remaining interior nodes. The resulting simultaneous equations can be expressed in matrix form as

F 0.73570 20.11438 20.2500020.11111 0.72222 20.11111 20.2500020.11438 0.73570 20.25000 20.31288 1.28888 20.14907

20.25000 20.11111 0.72222 20.11111

20.31288 20.14907 1.28888

V 3 fV1, 1V2, 1V3, 1

V1, 2 V2, 2 V3, 2

v 5 f121.320121.32 826.92

250

826.92

v which can be solved for

V1, 1 5 521.19  V2, 1 5 421.85  V3, 1 5 521.19

V1, 2 5 855.47  V2, 2 5 755.40  V3, 2 5 855.47

These results are depicted in Fig. 32.6a. To compute the fl ux (recall Sec. 29.2.3), Eqs. (29.14) and (29.15) must be modi- fi ed to account for the irregular boundaries. For the present example, the modifi cations

FIGURE 32.6 The results of solving the Laplace equation with correc- tion factors for the irregular boundaries. (a) Potential and (b) fl ux.

(a)

(b)

1000

1000

10001000

1000

1000 1000

0 00

855 755 855

521 422 521

cha9792x_ch32_915-930.indd Page 923 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 923 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

924 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

result in

Dx 5 2e Vi11, j 2 Vi21, j

(a1 1 a2) ¢x

and

Dy 5 2e Vi, j11 2 Vi, j21

(b1 1 b2)¢y

For node (1, 1), these formulas can be used to compute the x and y components of the fl ux

Dx 5 22 421.85 2 1000

(0.94281 1 1)3 5 198.4

and

Dy 5 22 855.47 2 0

(1 1 1)2 5 2427.7

which in turn can be used to calculate the electric fl ux density vector

D 5 2198.42 1 (2427.7)2 5 471.5

with a direction of

u 5 tan21 a2427.7 198.4

b 5 265.1° The results for the other nodes are

Node Dx Dy D U

2, 1 0.0 2377.7 377.7 290 3, 1 2198.4 2427.7 471.5 245.1 1, 2 109.4 2299.6 281.9 269.1 2, 2 0.0 2289.1 289.1 290.1 3, 2 2109.4 2299.6 318.6 249.9

The fl uxes are displayed in Fig. 32.6b.

32.4 FINITE-ELEMENT SOLUTION OF A SERIES OF SPRINGS (MECHANICAL/AEROSPACE ENGINEERING)

Background. Figure 32.7 shows a series of interconnected springs. One end is fi xed to a wall, whereas the other is subject to a constant force F. Using the step-by-step procedure outlined in Chap. 31, a fi nite-element approach can be employed to determine the displacements of the springs.

Solution.

Discretization. The way to partition this system is obviously to treat each spring as an element. Thus, the system consists of four elements and fi ve nodes (Fig. 32.7b).

cha9792x_ch32_915-930.indd Page 924 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 924 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.4 FINITE-ELEMENT SOLUTION OF A SERIES OF SPRINGS 925

Element equations. Because this system is so simple, its element equations can be written directly without recourse to mathematical approximations. This is an example of the direct approach for deriving elements. Figure 32.8 shows an individual element. The relationship between force F and displacement x can be represented mathematically by Hooke’s law:

F 5 kx

where k 5 the spring constant, which can be interpreted as the force required to cause a unit displacement. If a force F1 is applied at node 1, the following force balance must hold:

F 5 k(x1 2 x2)

where x1 5 displacement of node 1 from its equilibrium position and x2 5 displacement of node 2 from its equilibrium position. Thus, x2 2 x1 represents how much the spring is elongated or compressed relative to equilibrium (Fig. 32.8). This equation can also be written as

F1 5 kx1 2 kx2

For a stationary system, a force balance also necessitates that F1 5 2F2 and, therefore,

F2 5 2kx1 1 kx2

FIGURE 32.7 (a) A series of interconnected springs. One end is fi xed to a wall, whereas the other is sub- ject to a constant force F. (b) The fi nite-element representa- tion. Each spring represents an element. Therefore, the system consists of four elements and fi ve nodes.

1 2 3 4 5

1 2 3 4

Force

(a)

(b)

Node

Element

FIGURE 32.8 A free-body diagram of a spring system.

Node 1 Node 2

F1 F2

0 x1 x2 x

cha9792x_ch32_915-930.indd Page 925 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 925 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

926 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

These two simultaneous equations specify the behavior of the element in response to prescribed forces. They can be written in matrix form as

c k 2k 2k k

d e x1 x2 f 5 eF1

F2 f

or

[k]{x} 5 {F} (32.18)

where the matrix [k] is the element property matrix. For this case, it is also referred to as the element stiffness matrix. Notice that Eq. (32.18) has been cast in the format of Eq. (31.9). Thus, we have succeeded in generating a matrix equation that describes the behavior of a typical element in our system. Before proceeding to the next step—the assembly of the total solution—we will introduce some notation. The elements of [k] and {F} are conventionally superscripted and subscripted, as in

c k(e)11 2k(e)12 2k(e)21 k

(e) 22

d e x1 x2 f 5 eF(e)1

F (e)2 f

where the superscript (e) designates that these are the element equations. The k’s are also subscripted as kij to denote their location in the ith row and jth column of the matrix. For the present case, they can also be physically interpreted as representing the force required at node i to induce a unit displacement at node j.

Assembly. Before the element equations are assembled, all the elements and nodes must be numbered. This global numbering scheme specifi es a system confi guration or topology (note that the present case uses a scheme identical to Table 31.1). That is, it documents which nodes belong to which element. Once the topology is specifi ed, the equations for each element can be written with reference to the global coordinates. The element equations can then be added one at a time to assemble the total system. The fi nal result can be expressed in matrix form as [recall Eq. (31.10)]

[k]{x¿} 5 {F¿}

where

[k] 5 E k(1)11 2k(1)122k(1)21 k(1)22 1 k(2)11 2k(2)122k(2)21 k(2)22 1 k(3)11 2k(3)12 2k(3)21 k

(3) 22 1 k

(4) 11 2k

(4) 12

2k(4)21 k (4) 22

U (32.19) and

{F¿} 5 eF (1)100 0

F (4)2

u

cha9792x_ch32_915-930.indd Page 926 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 926 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

32.4 FINITE-ELEMENT SOLUTION OF A SERIES OF SPRINGS 927

and {x9} and {F9} are the expanded displacement and force vectors, respectively. Notice that, as the equations were assembled, the internal forces cancel. Thus, the fi nal result for {F9} has zeros for all but the fi rst and last nodes. Before proceeding to the next step, we must comment on the structure of the assem- blage property matrix [Eq. (32.19)]. Notice that the matrix is tridiagonal. This is a direct result of the particular global numbering scheme that was chosen (Table 31.1) prior to assemblage. Although it is not very important in the present context, the attainment of such a banded, sparse system can be a decided advantage for more complicated problem settings. This is due to the effi cient schemes that are available for solving such systems.

Boundary Conditions. The present system is subject to a single boundary condition, x1 5 0. Introduction of this condition and applying the global renumbering scheme re- duces the system to (k9s 5 1)D 2 2121 2 21

21 2 21

21 1

T dx2x3 x4 x5

t 5 d00 0

F

t The system is now in the form of Eq. (31.11) and is ready to be solved. Although reduction of the equations is certainly a valid approach for incorporating boundary conditions, it is usually preferable to leave the number of equations intact when performing the solution on the computer. Whatever the method, once the boundary con- ditions are incorporated, we can proceed to the next step—the solution.

Generating Solution. Using one of the approaches from Part Three, such as the effi - cient tridiagonal solution technique delineated in Chap. 11, the system can be solved for (with all k9s 5 1 and F 5 1)

x2 5 1  x3 5 2  x4 5 3  x5 5 4

Postprocessing. The results can now be displayed graphically. As in Fig. 32.9, the results are as expected. Each spring is elongated a unit displacement.

FIGURE 32.9 (a) The original spring system. (b) The system after the application of a constant force. The dis- placements are indicated in the space between the two systems.

F x = 1

x = 2 x = 3

x = 4

(a)

(b)

cha9792x_ch32_915-930.indd Page 927 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 927 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

928 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

PROBLEMS

Chemical/Bio Engineering 32.1 Perform the same computation as in Sec. 32.1, but use Dx 5 1.25. 32.2 Develop a fi nite-element solution for the steady-state system of Sec. 32.1. 32.3 Compute mass fl uxes for the steady-state solution of Sec. 32.1 using Fick’s fi rst law. 32.4 Compute the steady-state distribution of concentration for the tank shown in Fig. P32.4. The PDE governing this system is

D a 02c 0x2

1 02c 0y2 b 2 kc 5 0

and the boundary conditions are as shown. Employ a value of 0.5 for D and 0.1 for k. 32.5 Two plates are 10 cm apart, as shown in Fig. P32.5. Initially, both plates and the fl uid are still. At t 5 0, the top plate is moved at a constant velocity of 8 cm/s. The equations governing the motions of the fl uids are

0 yoil 0t

5 moil 02yoil 0x2  and 

0 ywater 0t

5 mwater 02ywater 0x2

and the following relationships hold true at the oil-water interface:

yoil 5 ywater and moil 0yoil 0x

5 mwater 0ywater 0x

What is the velocity of the two fl uid layers at t 5 0.5, 1, and 1.5 s at distances x 5 2, 4, 6, and 8 cm from the bottom plate? Note that mwater and moil 5 1 and 3 cp, respectively. 32.6 The displacement of a uniform membrane subject to a tension and a uniform pressure can be described by the Poisson equation

02z 0x2

1 02z 0y2

5 2 P

T

Solve for the displacement of a 1-cm-square membrane that has PyT 5 0.6/cm and is fastened so that it has zero displacement along its four boundaries. Employ Dx 5 Dy 5 0.1 cm. Display your re- sults as a contour plot.

Civil/Environmental Engineering 32.7 Perform the same computation as in Sec. 32.2, but use Dx 5 Dy 5 0.4 m. 32.8 The fl ow through porous media can be described by the Laplace equation

02h 0x2

1 02h 0y2

5 0

where h is head. Use numerical methods to determine the distribu- tion of head for the system shown in Fig. P32.8.

FIGURE P32.4

30

10

10

Open boundary Wall

c = 40

c = 100

FIGURE P32.5

Oil

Water

10

8

6

4

2

x = 0

FIGURE P32.8

2

= 0

12

�h �y

= 0

h = 20

�h �n

= 0�h �y

= 1�h �x

cha9792x_ch32_915-930.indd Page 928 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 928 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

PROBLEMS 929

32.9 The velocity of water fl ow through the porous media can be related to head by D’Arcy’s law

qn 5 2K dh

dn

where K is the hydraulic conductivity and qn is discharge velocity in the n direction. If K 5 5 3 1024 cm/s, compute the water ve- locities for Prob. 32.8.

Electrical Engineering 32.10 Perform the same computation as in Sec. 32.3 but for the system depicted in Fig. P32.10. 32.11 Perform the same computation as in Sec. 32.3 but for the system depicted in Fig. P32.11.

32.12 Use Poisson’s equation to compute the electric potential over a unit square (1 3 1) plate with zero voltage at the edges and point charge sources of ryye (0.7, 0.7) 5 1 and ryye (0.3, 0.3) 5 21. Employ Dx 5 Dy 5 0.1 and display your results as a contour plot.

Mechanical/Aerospace Engineering 32.13 Perform the same computation as in Sec. 32.4, but change the force to 1.5 and the spring constants to

Spring 1 2 3 4 k 0.75 1.5 0.5 2

32.14 Perform the same computation as in Sec. 32.4, but use a force of 2 and fi ve springs with

Spring 1 2 3 4 5 k 0.25 0.5 1.5 0.75 1

32.15 An insulated composite rod is formed of two parts arranged end to end, and both halves are of equal length. Part a has thermal conductivity ka, for 0 # x # 1y2, and part b has thermal conductiv- ity kb, for 1y2 # x # 1. The nondimensional transient heat conduc- tion equations that describe the temperature u over the length x of the composite rod are

02u 0x2

5 0u 0t   0 # x # 1y2

r 02u 0x2

5 0u 0t   1y2 # x # 1

where u 5 temperature, x 5 axial coordinate, t 5 time, and r 5 kaykb. The boundary and initial conditions are

Boundary conditions u(0, t ) 5 1 u(1, t ) 5 1

a 0u 0x b

a 5 a 0u 0x b

b x 5 1/2

Initial conditions u(x, 0) 5 0 0 , x , 1

Solve this set of equations for the temperature distribution as a func- tion of time. Use second-order accurate fi nite-difference analogues for the derivatives with a Crank-Nicolson formulation to integrate in time. Write a computer program for the solution, and select values of Dx and Dt for good accuracy. Plot the temperature u versus length x for various values of time t. Generate a separate curve for the follow- ing values of the parameter r 5 1, 0.1, 0.01, 0.001, and 0. 32.16 Solve the nondimensional transient heat conduction equa- tion in two dimensions, which represents the transient temperature distribution in an insulated plate. The governing equation is

02u 0x2

1 02u 0y2

5 0u 0t

FIGURE P32.10

FIGURE P32.11

a V = 10

V = 0

V = 5

V = 40

V = 20

a

V = 10

2

= 0

111

�V �y

= 0�V �y

= 0�V �x

V = 70

V = 100

cha9792x_ch32_915-930.indd Page 929 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 929 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

930 CASE STUDIES: PARTIAL DIFFERENTIAL EQUATIONS

where u 5 temperature, x and y are spatial coordinates, and t 5 time. The boundary and initial conditions are

Boundary conditions u(x, 0, t ) 5 0 u(x, 1, t ) 5 1

u(0, y, t ) 5 0 u(1, y, t ) 5 1

Initial condition u(x, y, 0) 5 0 0 # x , 1 0 # y , 1

Solve using the alternating direction-implicit technique. Write a computer program to implement the solution. Plot the results using a three-dimensional plotting routine where the horizontal plan con- tains the x and y axes and the z axis is the dependent variable u. Construct several plots at various times, including the following: (a) the initial conditions; (b) one intermediate time, approximately halfway to steady state; and (c) the steady-state condition.

cha9792x_ch32_915-930.indd Page 930 05/11/13 6:56 PM F-468 cha9792x_ch32_915-930.indd Page 930 05/11/13 6:56 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

27.1 CURRENT 1ST LEVEL HEAD 931

931

PT8.3 TRADE-OFFS

The primary trade-offs associated with numerical methods for the solution of partial differential equations involve choosing between fi nite-difference and fi nite-element ap- proaches. The fi nite-difference methods are conceptually easier to understand. In addi- tion, they are easy to program for systems that can be approximated with uniform grids. However, they are diffi cult to apply to systems with complicated geometries. Finite-difference approaches can be divided into categories depending on the type of PDE that is being solved. Elliptic PDEs can be approximated by a set of linear alge- braic equations. Consequently, the Liebmann method (which, in fact, is Gauss-Seidel) can be employed to obtain a solution iteratively. One-dimensional parabolic PDEs can be solved in two fundamentally different ways: explicit or implicit approaches. The explicit method steps out in time in a fashion that is similar to Euler’s technique for solving ODEs. It has the advantage that it is simple to program but has the shortcoming of a very stringent stability criterion. In contrast, stable implicit methods are available. These typically involve solving simultaneous tridiagonal algebraic equations at each time step. One of these approaches, the Crank-Nicolson method, is both accurate and stable and, therefore, is widely used for one-dimensional linear para- bolic problems. Two-dimensional parabolic PDEs can also be modeled explicitly. However, their stability constraints are even more severe than for the one-dimensional case. Special implicit approaches, which are generally referred to as splitting methods, have been developed to circumvent this shortcoming. These approaches are both effi cient and stable. One of the most common is the ADI, or alternating-direction implicit, method. All the above fi nite-difference approaches become unwieldy when applied to systems involving nonuniform shapes and heterogeneous conditions. Finite-element methods are available that handle such systems in a superior fashion. Although the fi nite-element method is based on some fairly straightforward ideas, the mechanics of generating a good fi nite-element code for two- and three-dimensional problems is not a trivial exercise. In addition, it can be computationally expensive for large problems. However, it is vastly superior to fi nite-difference approaches for systems involving complicated shapes. Consequently, its expense and conceptual “overhead” are often justifi ed because of the detail of the fi nal solution.

PT8.4 IMPORTANT RELATIONSHIPS AND FORMULAS

Table PT8.3 summarizes important information that was presented regarding the fi nite- difference methods in Part Eight. This table can be consulted to quickly access important relationships and formulas.

EPILOGUE: PART EIGHT

cha9792x_ep08_931-932.indd Page 931 05/11/13 6:57 PM F-468 cha9792x_ep08_931-932.indd Page 931 05/11/13 6:57 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

932 EPILOGUE: PART EIGHT

TABLE PT8.3 Summary of fi nite-difference methods.

Computational Molecule Equation

Elliptic PDEs

i – 1, j i, j i + 1, j

i, j + 1

i, j – 1

Ti,j 5 Ti11,j 1 Ti21,j 1 Ti,j11 1 Ti,j21

4 Liebmann’s method

Parabolic PDEs (one-dimensional) Explicit method

i – 1, l i, l i + 1, l

i, l + 1

T l11i 5 T li 1 l(T li11 2 2T li 1 T li21)

Implicit method

i – 1, l + 1

i, l

i + 1, l + 1i, l + 1

2lT l11i21 1 (1 1 2l)T l11i 2 lT l11i11 5 T li

Crank-Nicolson method

i – 1, l + 1

i – 1, l i, l

i + 1, l + 1

i + 1, l

i, l + 1

i, l + 1 2

2lT l11i21 1 2(1 1 l)T l11i 2 lT l11i11

5 lT li21 1 2(1 2 l)T li 1 lT li11

PT8.5 ADVANCED METHODS AND ADDITIONAL REFERENCES

Carnahan, Luther, and Wilkes (1969); Rice (1983); Ferziger (1981); and Lapidus and Pinder (1981) provide useful surveys of methods and software for solving PDEs. You can also consult Ames (1977), Gladwell and Wait (1979), Vichnevetsky (1981, 1982), and Zienkiewicz (1971) for more in-depth treatments. Additional information on the fi nite- element method can be found in Allaire (1985), Huebner and Thornton (1982), Stasa (1985), and Baker (1983). Aside from elliptic and hyperbolic PDEs, numerical methods are also available to solve hyperbolic equations. Nice introductions and summaries of some of these methods can be found in Lapidus and Pinder (1981), Ferziger (1981), Forsythe and Wasow (1960), and Hoffman (1992).

cha9792x_ep08_931-932.indd Page 932 05/11/13 6:57 PM F-468 cha9792x_ep08_931-932.indd Page 932 05/11/13 6:57 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

933

The Fourier Series

A A P P E N D I X A

The Fourier series can be expressed in a number of different formats. Two equivalent trigonometric expressions are

f(t) 5 a0 1 a q

k51 [ak cos (kv0t) 1 bk sin (kv0t)]

or

f(t) 5 a0 1 a q

k51 [ck cos (kv0t 1 uk)]

where the coeffi cients are related by (see Fig. A.1)

ck 5 2a 2 k 1 b

2 k

and

uk 5 2tan 21 abk

ak b

In addition to the trigonometric formats, the series can also be expressed in terms of the exponential function as

f(t) 5 c̃0 1 a q

k51 [c̃k e

ikv0t 1 c̃2k e 2ikv0 t] (A.1)

where (see Fig. A.2)

c̃0 5 a0

c̃k 5 1

2 (ak 2 ibk) 5 Z c̃k Z eifk

c̃2k 5 1

2 (ak 1 ibk) 5 Z c̃k Z e2ifk

FIGURE A.1 Relationships between rectangular and polar forms of the Fourier series coeffi cients.

ak

bk

–�k

a k + b k2

2

FIGURE A.2 Relationships between complex exponential and real coeffi cients of the Fourier series.

c–k �

ck

�k

�–k

ak 2

�ck� �

�ck� �bk

2 –

cha9792x_app_933-942.indd Page 933 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 933 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

934 APPENDIX A THE FOURIER SERIES

where Z c̃0 Z 5 a0 and

Z c̃k Z 5 1

2 2a2k 1 b

2 k 5

ck 2

and

fk 5 tan 21 a2bk

ak b

Note that the tilde signifi es that the coeffi cient is a complex number.

Each term in Eq. (A.1) can be visualized as a rotating phasor (the arrows in Fig. A.2). Terms with a positive sub- script rotate in a counterclockwise direction, whereas those with a negative subscript rotate clockwise. The coeffi cients c̃k and c̃2k specify the position of the phasor at t 5 0. The infinite summation of the spinning phasors, which are allowed to rotate at t 5 0, is then equal to f(t).

cha9792x_app_933-942.indd Page 934 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 934 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

935

Getting Started with MATLAB

MATLAB software is a computer program that provides the user with a convenient en- vironment for many types of calculations—in particular, those that are related to matrix manipulations. MATLAB operates interactively, executing the user’s command one-by-one as they are entered. A series of commands may be saved as a script and run like an interpretive program. MATLAB has a large number of built-in functions; however, it is possible for users to build their own functions made up of MATLAB commands and functions. The primary features of MATLAB are built-in vector and matrix computations including:

Vector-matrix arithmetic. Matrix inversion and eigenvalue/vector analysis. Complex arithmetic and polynomial operations. Statistical calculations. Graphical displays. Control system design. Fitting process models from test data.

MATLAB has a number of optional toolboxes that provide specialized functions. These include: signal processing, control systems, system identifi cation, optimization, and statistics. MATLAB is available in versions that run on PCs, Macs, and workstations. The modern version that runs on PCs does so in the Windows environment. The seven exer- cises that follow are meant to give you the fl avor of computing with MATLAB; they do not constitute a comprehensive tutorial. There are additional tutorial materials in the MATLAB manuals. A number of textbooks now feature MATLAB exercises. Also, on- line information is available for any command or function by typing: help name, where name identifi es the command. Do not just look through these exercises; try them all and try variations that occur to you. Check the answers that MATLAB gives and make sure you understand them and they are correct. That is the effective way to learn MATLAB.

B A P P E N D I X B

cha9792x_app_933-942.indd Page 935 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 935 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

936 APPENDIX B GETTING STARTED WITH MATLAB

1. Assignment of Values to Variable Names

Assignment of values to scalar variables is similar to other computer languages. Try typing

a = 4

and

A = 6

Note how the assignment echoes to confi rm what you have done. This is a characteristic of MATLAB. The echo can be suppressed by terminating the command line with the semicolon (;) character. Try typing

b = −3;

MATLAB treats names in a case-sensitive manner, that is, the name a is not the same as the name A. To illustrate this, enter

a

and

A

See how their values are distinct. They are distinct names. Variable names in MATLAB generally represent matrix quantities. A row vector can be assigned as follows:

a = [1 2 3 4 5]

The echo confi rms the assignment again. Notice how the new assignment of a has taken over. A column vector can be entered in several ways. Try them.

b = [ 1 ; 2 ; 3 ; 4 ; 5 ]

or

b = [ 1; 2; 3; 4; 5 ]

or, by transposing a row vector with the ' operator,

b = [ 1 2 3 4 5 ]'

A two-dimensional matrix of values can be assigned as follows:

A = [ 1 2 3 ; 4 5 6 ; 7 8 8 ]

or

A = [ 1 2 3 ; 4 5 6 ; 7 8 8 ]

cha9792x_app_933-942.indd Page 936 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 936 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX B GETTING STARTED WITH MATLAB 937

The values stored by a variable can be examined at any time by typing the name alone, for example,

b

or

A

Also, a list of all current variables can be obtained by entering the command

who

or, with more detail, enter

whos

There are several predefi ned variables, for example, pi. It is also possible to assign complex values to variables, since MATLAB handles complex arithmetic automatically. To do this, it is convenient to assign a variable name, usually either i or j, to the square root of 21.

i = sqrt(−1)

Then, a complex value can be assigned, like

x = 2 + i*4

2. Mathematical Operations

Operations with scalar quantities are handled in a straightforward manner, similar to computer languages. The common operators, in order of priority, are

Exponentiation * / Multiplication and division \ Left division (applies to matrices) + − Addition and subtraction

These operators will work in calculator fashion. Try

2 * pi

Also, scalar real variables can be included:

y = pi / 4 y 2.45

Results of calculations can be assigned to a variable, as in the next-to-last example, or simply displayed, as in the last example. Calculations can also involve complex quantities. Using the x defi ned above, try

3 * x 1 / x x 2 x + y

cha9792x_app_933-942.indd Page 937 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 937 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

938 APPENDIX B GETTING STARTED WITH MATLAB

The real power of MATLAB is illustrated in its ability to carry out matrix calculations. The inner product of two vectors (dot product) can be calculated using the * operator,

a * b

and likewise, the outer product

b * a

To illustrate vector-matrix multiplication, fi rst redefi ne a and b,

a = [1 2 3]

and

b = [4 5 6]'

Now, try

a * A

or

A * b

What happens when the dimensions are not those required by the operations? Try

A * a

Matrix-matrix multiplication is carried out in likewise fashion:

A * A

Mixed operations with scalars are also possible:

A / pi

It is important to always remember that MATLAB will apply the simple arithmetic op- erators in vector-matrix fashion if possible. At times, you will want to carry out calcula- tions item-by-item in a matrix or vector. MATLAB provides for that too. For example,

A 2

results in matrix multiplication of A with itself. What if you want to square each element of A? That can be done with

A . 2

The ? preceding the operator signifi es that the operation is to be carried out item-by- item. The MATLAB manual calls these array operations. When the division operator (/) is used with matrices, the use of a matrix inverse is im- plied. Therefore, if A is a square, nonsingular matrix, then b/A corresponds to the right multiplication of b by the inverse of A. A longer way to do this used the inv function, that is, b*inv(A) ; however, using the division operator is more effi cient since X = b/A is actually solved as the set of equations X*A=b using a decomposition/elimination scheme. The “left division” operator (\ , the backslash character) is used in matrix op- erations also. As above, A\b corresponds to the left multiplication of b by the inverse

cha9792x_app_933-942.indd Page 938 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 938 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX B GETTING STARTED WITH MATLAB 939

of A. This is actually solved as the set of equations A*X=b , a common engineering calculation. For example, if c is a column vector with values 0.1, 1.0, and 10, the solution of A * x = c , where A has been set above, can be obtained by typing

c = [0.1 1.0 10]' x = A\c

Try that.

3. Use of Built-In Functions

MATLAB and its Toolboxes have a rich collection of built-in functions. You can use on-line help to fi nd out more about them. One of their important properties is that they will operate directly on vector and matrix quantities. For example, try

log(A)

and you will see that the natural logarithm function is applied in array style, element by element, to the matrix A. Most functions, like sqrt, abs, sin, acos, tanh, and exp, operate in array fashion. Certain functions, like exponential and square root, have matrix defi ni- tions also. MATLAB will evaluate the matrix version when the letter m is appended to the function name. Try

sqrtm(A)

A common use of functions is to evaluate a formula for a series of arguments. Create a column vector t that contains values from 0 to 100 in steps of 5,

t = [100]'

Check the number of items in the t array with the Length function,

length(t)

Now, say that you want to evaluate a formula y 5 f(t), where the formula is computed for each value of the t array, and the result is assigned to a corresponding position in the y array. For example,

y = t . 0.34 − log10(t) + 1 ./ t

Done! [Note the use of the array operators adjacent decimal points.] This is similar to creating a column of the t values on a spreadsheet and copying a formula down an adjacent column to evaluate y values.

4. Graphics

MATLAB’s graphics capabilities have similarities to those of a spreadsheet program. Graphs can be created quickly and conveniently; however, there is not much fl exibility to customize them. For example, to create a graph of the t,y arrays from the data above, enter

plot(t, y)

cha9792x_app_933-942.indd Page 939 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 939 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

940 APPENDIX B GETTING STARTED WITH MATLAB

That’s it! You can customize the graph a bit with commands like the following:

title('Plot of y versus t') xlabel('Values of t') ylabel('Values of y') grid

The graph appears in a separate window and can be printed or transferred via the clipboard (PCs with Windows or Macs) to other programs. There are other features of graphics that are useful, for example, plotting objects instead of lines, families of curves plots, plotting on the complex plane, multiple graphs windows, log-log or semilog plots, three-dimensional mesh plots, and contour plots.

5. Polynomials

There are many MATLAB functions that allow you to operate on arrays as if their entries were coeffi cients or roots of polynomial equations. For example, enter

c = [1 1 1 1]

and then

r = roots(c)

and the roots of the polynomial x3 1 x2 1 x 1 1 5 0 should be printed and are also stored in the r array. The polynomial coeffi cients can be computed from the roots with the poly function,

poly(r)

and a polynomial can be evaluated for a given value of x. For example,

polyval(c, 1.32)

If another polynomial, 2x2 2 0.4x 2 1, is represented by the array d,

d = [2 −0.4 −1]

the two polynomials can be multiplied symbolically with the convolution function, conv, to yield the coeffi cients of the product polynomial,

cd = conv(c,d)

The deconvolution function, deconv, can be used to divide one polynomial into another, for example,

[q,r] = deconv(c,d)

The q result is the quotient, and the r result is the remainder. There are other polynomial functions that may become useful to you, such as the residue function for partial fraction expansion.

6. Statistical Analysis

The Statistics Toolbox contains many features for statistical analysis; however, common statistical calculations can be performed with MATLAB’s basic function set. You can

cha9792x_app_933-942.indd Page 940 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 940 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX B GETTING STARTED WITH MATLAB 941

generate a series of (pseudo) random numbers with the rand function. Either a uniform (rand) or normal (randn) distribution is available:

n = 0:5:1000;

(Did you forget the ; !!!)

num = randn(size(n));

You probably understand why using the semicolon at the end of the commands above is important, especially if you neglected to do so. If you would like to see a plot of noise, try

plot(num)

These are supposed to be normally distributed numbers with a mean of zero and variance (and standard deviation) of one. Check by

mean(num)

and

std(num)

No one is perfect! You can fi nd minimum and maximum values,

min(num) max(num)

There is a convenient function for plotting a histogram of the data:

hist(num,20)

where 20 is the number of bins. If you would like to fi t a polynomial to some data by least squares, you can use the polyfi t function. Try the following example:

t = 05 y = [−0.45 0.56 2.34 5.6 9.45 24.59] coef = polyfit(t, y, 3)

The values in coef are the fi tted polynomial coeffi cients. To generate the computed value of y,

yc 5 polyval(coef,t)

and to plot the data versus the fi tted curve,

plot (t,yc,t,y,'o')

The plot of the continuous curve is piecewise linear; therefore, it does not look very smooth. Improve this as follows:

t1 = [0 : 0.05 : 5]; yc = polyval(coef, t1); plot(t1, yc, t, y, 'o')

cha9792x_app_933-942.indd Page 941 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 941 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

942 APPENDIX B GETTING STARTED WITH MATLAB

7. This and That

There are many, many other features to MATLAB. Some of these you will fi nd useful; perhaps others you will never use. We encourage you to explore and experiment. To save a copy of your session, MATLAB has a useful capability called diary. You issue the command

diary problem1

and MATLAB opens a disk fi le in which it stores all the subsequent commands and results (not graphs) of your session. You can turn the diary feature off:

diary off

and back on with the same fi le:

diary on

After you leave MATLAB, the diary fi le is available to you. It is common to use an editor or word processor to clean up the diary fi le (getting rid of all those errors you made before anyone can see them!) and then print the fi le to obtain a hard copy of the important parts of your work session, for example, key numerical results. Exit MATLAB with the quit or exit commands. It is possible to save the current state of your work with the save command. It is also possible to reload that state with the load command.

cha9792x_app_933-942.indd Page 942 19/11/13 6:50 PM F-468 cha9792x_app_933-942.indd Page 942 19/11/13 6:50 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

943

Getting Started with Mathcad

Mathcad has a unique way to handle equations, numbers, text, and graphs. It works like a scratch pad and pencil. The screen interface is a blank worksheet on which you can enter equations, graph data or functions, and annotate operations with text. It uses stan- dard mathematical symbols to represent operators when possible. Therefore, you may fi nd that the Mathcad interface is quite natural and familiar. Mathcad can solve a wide range of mathematical problems either numerically or symbolically. The symbolic capabilities of Mathcad have relatively little application in this text, although they may be used to check our numerical results. Therefore, they will not be covered in detail in this overview. Mathcad has a comprehensive set of operators and functions that allow you to perform many of the numerical methods covered in this text. It also has a programming language that allows you to write your own multiline procedures and subprograms. The following discussion provides a brief description of the features of Mathcad you will fi nd most useful for this text.

THE BASICS OF MATHCAD

Applications in this text will require that you be able to create your own worksheets. To facilitate your efforts, let’s go over the main features of the Mathcad application window.

The Main Menu

This is your gateway to Mathcad. It also provides commands that handle the details of editing and managing your worksheets. For example, click on the File and Tools menus to see some of the functionality available to you.

The Standard Toolbar

Several toolbars should be automatically displayed just below the Main menu. As the name implies, the Standard toolbar provides shortcuts for many common tasks, from worksheet opening and fi le saving to bringing up lists of built-in functions and units. Depending on what you are doing in your worksheet, one or more of these buttons may

C A P P E N D I X C

cha9792x_appC_943-953.indd Page 943 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 943 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

944 APPENDIX C GETTING STARTED WITH MATHCAD

appear grayed out. If you let your mouse hover over each of the buttons on the palette, you will see a description of the button’s function.

The Math Palette

The Math Palette may automatically be displayed at the top of the screen. If not, just select View, Toolbars, Math and it will appear. The buttons and their functions are described below:

C alculator Boolean

Gr aph P rogramming

Vector and Matr ix Gr eek Symbol

Ev aluation S ymbolic Keyword

C alculus

Click on one of these buttons to bring up the full palette. You can use the palettes to insert math symbols and operations directly into your Mathcad worksheet.

ENTERING TEXT AND MATHEMATICAL OPERATIONS

Entering Text

To create a text region, click in a blank area of the screen to position the red crosshair cursor and type a double quote ["]. Now you can type whatever you like, just as in a word processor. As the region grows, a black box appears around the text. The box has resizing “handles” on the right and bottom edges of the rectangle. Once you are done, click outside the text region to go back to inputting math operations. The black selection box disappears when you are no longer working in the text region.

Mathematical Operations

Type See on Screen

11

Click somewhere in the upper-left-hand corner of the worksheet, and the red crosshair should move to where you click. After you type the number 1 and the 1 sign you will see a little black box delimited by blue editing lines. In Mathcad this black box is called a

cha9792x_appC_943-953.indd Page 944 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 944 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX C GETTING STARTED WITH MATHCAD 945

placeholder. If you continue typing, whatever you type next will appear in the placeholder. For example, type 2 in the placeholder, then press the equals key ( 5 ) to see the result.

1 1 2 5 3

The basic arithmetic operators are listed below, along with their keystrokes and Calculator Palette button equivalents.

Operation Keystroke Palette Example

Addition 1 1 2 1 2 5 4 Subtraction 2 2 2 2 2 5 0 Multiplication * 3 2 ? 2 5 4 Division / 4 22 5 1 Exponentiation ^ xY 22 5 4

Notice that operations in a Mathcad worksheet appear in familiar notation—multiplication as a dot, division with a fraction bar, exponents in a raised position, and so on. Calculations are computed internally to 15 places, but you can show fewer places in the answer. To change the default display of numerical and symbolic results in a worksheet, click in a blank area of the worksheet. Then select Result from the Format menu to display the Result Format dialog box, and choose your default settings. Make sure that the button labeled “Set as default” is checked, and click OK. If you just want to change the display of a particular result, click on the equation, and follow the same steps. Here are a few more examples that demonstrate Mathcad features.

B

1.837 # 103

100 1 35 5 2.3142353232

Most standard engineering and mathematical functions are built in.

log(1347.2) # sin a3 5

# pb 5 2.976 Mathcad’s functions and operators easily handle complex numbers.

(2.3 1 4.7i)3 1 e322i 5 2148.613 2 47.498i

MATHEMATICAL FUNCTIONS AND VARIABLES

The defi nition symbol :5 is used to defi ne a function or variable in Mathcad. For example, click an empty worksheet to position the red crosshair in a blank area and type:

Type See on Screen

f(x):x^2

cha9792x_appC_943-953.indd Page 945 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 945 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

946 APPENDIX C GETTING STARTED WITH MATHCAD

The defi nition symbol is also located on the Evaluation selection of the Math Palette. When you change a defi nition function or variable, Mathcad immediately recalculates any new values that depend on it. Once you’ve defi ned a function like f(x), you can use it in a number of ways, for example:

f(x) :5 x2

Now you can insert a numerical value as the argument of f(x)

f(10) 5 100

or defi ne a variable and insert it as the argument of f(x).

x :5 3

f(x) 5 9

You can even defi ne another function in terms of f(x).

g(y) :5 f(y) 1 6

g(x) 5 15

Note that you can defi ne a function using expressions you build up from the keyboard or from the palettes of math operators. You can also include any of Mathcad’s hundreds of built-in functions. To see a list of built-in functions along with brief descriptions, select Function from the Insert menu, or click on the f(x) button. You can also type the name of any built-in function directly from the keyboard. The following are just a few examples that use some of Mathcad’s built-in functions.

Trig and Logs

ln(26) 5 3.258  csc(45 # .deg) 5 1.414

Matrix Functions

identity(3) 5 £ 1 0 0

0 1 0

0 0 1

§

Probability Distributions

pnorm(2,0,1) 5 0.977

Range Variables

In Mathcad you will fi nd yourself wanting to work with a range of values for many applications—for example, to defi ne a series of values to plot. Mathcad therefore provides the range operator ( .. ), which can be entered by typing a semicolon ( ; ) at the keyboard.

cha9792x_appC_943-953.indd Page 946 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 946 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX C GETTING STARTED WITH MATHCAD 947

The fi rst and last numbers establish the endpoints of the range variable, and the second number sets the increment. For example,

Type See on Screen

z:0,0.5;2 z:50,0.5..2

z 5

Matrix Computations and Operations

To enter a matrix, click on the 3 3 3 matrix icon in the Matrix Palette (or choose Matrix from the Insert menu), choose the number of rows and columns, then fi ll in the placeholders. For example,

A :5 £ 4 5 1

5 0 212

27 2 8

§

To compute the inverse,

Type See on Screen

A^–15 A21 5 £ 0.074 20.117 20.184

0.135 0.12 0.163

0.031 20.132 20.077

§

Mathcad has a comprehensive set of commands to perform various matrix operations. For example, to fi nd the determinant, type a vertical bar ( Z ) or use the button on the Matrix Palette.

ZA Z 5 326

Units

Mathcad can also handle units. To see the built-in units, choose Unit from the Insert menu, or click on the appropriate toolbar button. Let’s start with a simple example. Open a new worksheet in Mathcad and type Mass:75kg[Enter]. You should see

Mass :5 75 kg

cha9792x_appC_943-953.indd Page 947 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 947 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

948 APPENDIX C GETTING STARTED WITH MATHCAD

You could also have typed Mass:75*kg[Enter], multiplying the quantity times the unit. Now, enter g:9.8m/sˆ2 and you should have

g :5 9.8 m

s2

To see how Mathcad manages the units with calculations, enter Mass*g= and the result should be displayed with combined units as

Mass # g 5 735 N

Mathcad uses the SI unit system by default, but you can change that from Tools, Worksheet Options, Unit System. Alternate systems include CGS, MKS, and US. In- stead of typing in the unit, you can also insert it from a list. Try the following. Type Temp:273.16* then click on Insert, Unit. Select Temperature from the upper box and Kelvin(K) from the lower box and click OK.

NUMERICAL METHODS FUNCTION

Mathcad has a number of special built-in functions that perform a variety of numerical operations of particular interest to readers of this book. Examples of the development and application of these functions are described in detail in the text. Here we will provide a brief list of some of the more important functions just to give you an overview of the capabilities. We illustrate their use in the relevant sections of this book.

Function Name Use

root Solves f(x) 5 0 polyroots Finds all roots of a polynomial fi nd Solves a system of nonlinear algebraic equations minerr Returns a minimum error solution of a system of equations lsolve Solves a system of linear algebraic equations linterp Linear interpolation cspline Cubic spline interpolation regress Polynomial regression genfi t General nonlinear regression fft Fourier transform ifft Inverse Fourier transform rkfi xed Solves a system of differential equations using a fi xed step-size fourth-

order Runge-Kutta method rkadapt Solves a system of differential equations using a variable step-size

fourth-order Runge-Kutta method sbval Solves a two-point boundary value problem eigenvals Finds eigenvalues eigenvecs Finds eigenvectors relax Solves Poisson’s equation for a square domain

cha9792x_appC_943-953.indd Page 948 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 948 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX C GETTING STARTED WITH MATHCAD 949

MULTILINE PROCEDURES AND SUBPROGRAMS

The Programming Palette in Mathcad provides the capability for multiline procedures or subprograms with standard control structures such as FOR and WHILE loops, branch- ing, recursion, and more. Subprograms can be integrated with Mathcad’s worksheets and can operate on scalars, vectors, arrays, and even arrays of arrays.

CREATING GRAPHS

Mathcad’s graphics capabilities are particularly important to engineering work. The fi rst type of graph to know about is the QuickPlot. Start with a new worksheet and enter the following formula. Leave the formula selected.

e 2X

4  # (2 2 x) 2 1 Z

From the menu, select Insert, Graph, X-Y Plot. Click away from the graph and you should see an automatic QuickPlot.

This plot can now be adjusted as desired. For example, click in the plot to select it and change the x limits to 0 to 1. This should appear as

Then, click away from the plot, and the y axis will rescale automatically. Double-click on the plot, and the Formatting Currently Selected X-Y Plot dialog box should appear. Check the boxes for X and Y gridlines and click OK. Click away from the plot, and you should now have

cha9792x_appC_943-953.indd Page 949 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 949 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

950 APPENDIX C GETTING STARTED WITH MATHCAD

Mathcad graphs one point for each value of the range variable x. This variable was created automatically in this case. The x-y points are joined by short straight-line seg- ments. You can create your own range variables for the x axis. Instead of using a formula for the y axis, you can use a function. To illustrate this, enter the following function defi nition above your chart:

and change the y axis from the formula to f(x). You should have the same plot, but now it is in terms of the function f(x) instead of the direct formula. A function can also be used for the x axis. Try another example below your current graph. Make the following defi nitions:

N :5 100     u :5 0, 2 # x N

.. 2 # x

x(u) :5 cos(u)  y(u) :5 sin(u)

Insert a blank plot by pressing the @ key (Shift-2). Enter y (u) in the y-axis placeholder and x(u) in the x-axis placeholder and click away from the graph. This should yield the plot of a circle:

cha9792x_appC_943-953.indd Page 950 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 950 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX C GETTING STARTED WITH MATHCAD 951

It is also possible to plot the elements of a vector. Create the following vector of binomial probabilities:

i :50.. 10 pi :50.4

i.0.6(102 i)

and insert a graph with i on the x axis and pi on the y axis (you can use the [ key for the subscript). Your graph should look like

Of course, you can plot one vector against another too, as long as they have the same number of elements. The vectors could contain data instead a mathematical formula. There are many axis settings that can be adjusted by double-clicking on the graph. These are self-explanatory to a great extent, and you can become familiar with them through practice. There are many other styles of plots that can be generated by Mathcad. These include polar, surface, contour, 3D bar, scatter, vector-fi eld plots, and graphical animations.

SYMBOLIC MATHEMATICS

An intriguing and valuable feature of Mathcad is its capability to carry out symbolic math manipulations. The symbolic capabilities include

Algebraic manipulations. Calculus: differentiation and integration. Solving algebraic equations and systems of such equations.

and, more advanced features

Symbolic Fourier, Laplace, and z transforms. Symbolic optimization.

We will review the fi rst group here. Let’s start with a simple example of symbolic algebra. Enter the expression

(x 1 2) # (x 2 1) # (x 1 4)

Expand the horizontal editing line to encompass the entire expression and then select Expand from the Symbolics menu. You should see below:

x3 1 5 # x2 1 2 # x 2 8

cha9792x_appC_943-953.indd Page 951 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 951 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

952 APPENDIX C GETTING STARTED WITH MATHCAD

Now, enter the polynomial x3 1 3x2 1 3x 1 1 and follow the same procedure, except pick Factor from the Symbolics menu. You should have

x3 1 3 # x2 1 3 # x 1 1 (x 1 1)3

Another way to carry out symbolic commands is with keystrokes. Enter the following expression:

x2 2 3 # x 2 4 x 2 4

1 2 # x 2 5 Z

Then press the Ctrl-Shift-. key combination and type simplify into the placeholder that appears. You should get

x2 2 3 # x 2 4 x 2 4

1 2 # x 2 5 simplify S 3 # x 2 4

This is a different style of symbolic evaluation with the keyword retained and the result appearing out to the right. You’ve seen the use of three important symbolic operators:

expand Expand all powers and products of sums. factor Factor into a product of simpler functions. simplify Simplify by performing arithmetic, canceling common factors, using

identities, and simplifying powers.

Additional algebraic features include expansion to a series, partial fraction expansion, and extracting coeffi cients of a polynomial into a vector. Now, let’s experiment with simple differentiation. Enter the expression

and leave the T selected (or click on it to select it). Then select Variable and Differentiate from the Symbolics menu. You should have the result

Another way to do this is to use the differentiation operator from the Calculus toolbar. Enter the following expression:

cha9792x_appC_943-953.indd Page 952 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 952 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

APPENDIX C GETTING STARTED WITH MATHCAD 953

and then click on the button on the Evaluation toolbar (you can also press Ctrl-.). The result should be

Symbolic integrals can be determined either in indefi nite or defi nite form. For an

indefi nite integral, start by typing Ctrl-i or click on the button on the Calculus

toolbar. Then enter the desired function and differential followed by the button to produce the following result:

Mathcad can also compute limits symbolically. The appropriate buttons are on the Calculus toolbar.

LEARNING MORE ABOUT MATHCAD

In this brief introduction we have covered only the Mathcad basics. Further help is avail- able right in the Mathcad software package in a variety of forms.

ToolTips

Let your mouse pointer hover over a palette or toolbar button for a few seconds. You will see an explanatory tooltip displayed near the button. Look also on the message line at the bottom of the Mathcad application window for helpful tips and shortcuts.

Resource Center and QuickSheets

To help you get going fast and keep you learning, Mathcad comes complete with Quick- Sheets. These provide mathematical shortcuts for frequently used analyses—from graph- ing a function to solving simultaneous equations to the analysis of variance. There are numerous QuickSheets. To open the QuickSheets section, choose QuickSheets from the Help option on the Main menu.

Online Help

Online Help provides detailed, step-by-step instructions for using all of Mathcad’s features. Help is available at any time by simply going to the Help button on the Main menu. There you will fi nd several links including the Mathcad website and Mathcad training.

cha9792x_appC_943-953.indd Page 953 25/11/13 3:53 PM F-468 cha9792x_appC_943-953.indd Page 953 25/11/13 3:53 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

954

BIBLIOGRAPHY

Al-Khafaji, A. W., and J. R. Tooley, Numerical Methods in Engineering Practice, Holt, Rinehart and Winston, New York, 1986.

Allaire, P. E., Basics of the Finite Element Method, William C. Brown, Dubuque, IA, 1985.

Ames, W. F., Numerical Methods for Partial Differential Equations, Academic Press, New York, 1977.

Ang, A. H-S., and W. H. Tang, Probability Concepts in Engineering Planning and Design, Vol. 1: Basic Principles, Wiley, New York, 1975.

APHA (American Public Health Association). 1992. Standard Methods for the Examination of Water and Wastewater, 18th ed., Washington , DC.

Atkinson, K. E., An Introduction to Numerical Analysis, Wiley, New York, 1978.

Atkinson, L. V., and P. J. Harley, An Introduction to Numerical Methods with Pascal, Addison-Wesley, Reading, MA, 1983.

Baker, A. J., Finite Element Computational Fluid Mechanics, McGraw-Hill, New York, 1983.

Bathe, K.-J., and E. L. Wilson, Numerical Methods in Finite Element Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1976.

Booth, G. W., and T. L. Peterson, “Nonlinear Estimation,” I.B.M. Share Program Pa. No. 687 WLNL1, 1958.

Boyce, W. E., and R. C. DiPrima, Elementary Differential Equations and Boundary Value Problems, 5th ed. Wiley, New York, 1992.

Branscomb, L. M., “Electronics and Computers: An Overview,” Science, 215:755, 1982.

Brent, R. P., Algorithms for Minimization Without Derivatives , Prentice-Hall, Englewood Cliffs, NJ, 1973.

Brigham, E. O., The Fast Fourier Transform, Prentice-Hall, Englewood Cliffs, NJ, 1974.

Burden, R. L., and J. D. Faires, Numerical Analysis, 8th ed., PWS Publishing, Boston, 2005.

Butcher, J. C., “On Runge-Kutta Processes of Higher Order,” J. Austral. Math. Soc., 4:179, 1964.

Carnahan, B., H. A. Luther, and J. O. Wilkes, Applied Numerical Methods, Wiley, New York, 1969.

Cash, J. R., and A. H. Karp, ACM Transactions on Mathematical Software, 16:201–222, 1990.

Chapra, S. C., Surface Water-Quality Modeling, McGraw-Hill, New York, 1997.

Chapra, S. C., Applied Numerical Methods with MATLAB, 2nd ed., McGraw-Hill, New York, 2007.

Cheney, W., and D. Kincaid, Numerical Mathematics and Computing, 6th ed., Brooks/Cole, Monterey, CA, 2008.

Chirlian, P. M., Basic Network Theory, McGraw-Hill, New York, 1969.

Cooley, J. W., P. A. W. Lewis, and P. D. Welch, “Historical Notes on the Fast Fourier Transform,” IEEE Trans. Audio Electroacoust., AU-15(2):76–79, 1977.

Dantzig, G. B., Linear Programming and Extensions, Princeton University Press, Princeton, NJ, 1963.

Davis, H. T., Introduction to Nonlinear Differential and Integral Equations, Dover, New York, 1962.

Davis, L., Handbook of Genetic Algorithms, Van Nostrand Reinhold, New York, 1991.

Davis, P. J., and P. Rabinowitz, Methods of Numerical Integration, Academic Press, New York, 1975.

Dennis, J. E., and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1996.

Dekker, T. J., “Finding a Zero by Means of Successive Linear Interpolation , ” in Constructive Aspects of the Fundamental Theorem of Algebra , B. Dejon and P. Henrici (editors), Wiley-Interscience, New York, 1969, pp. 37–48.

Dijkstra, E. W., “Go To Statement Considered Harmful,” Commun. ACM, 11(3):147–148, 1968.

Draper, N. R., and H. Smith, Applied Regression Analysis, 2d ed., Wiley, New York, 1981.

Enright, W. H., T. E. Hull, and B. Lindberg, “Comparing Numerical Methods for Stiff Systems of ODE’s,” BIT, 15:10, 1975.

Fadeev, D. K., and V. N. Fadeeva, Computational Methods of Linear Algebra, Freeman, San Francisco, 1963.

Ferziger, J. H., Numerical Methods for Engineering Application, Wiley, New York, 1981.

Fletcher, R., Practical Methods of Optimization: 1: Unconstrained Optimization, Wiley, Chichester, UK, 1980.

Fletcher, R., Practical Methods of Optimization: 2: Constrained Optimization, Wiley, Chichester, 1981.

Forsythe, G. E., and W. R. Wasow, Finite-Difference Methods for Partial Differential Equations, Wiley, New York, 1960.

cha9792x_bib_954-956.indd Page 954 19/11/13 6:55 PM F-468 cha9792x_bib_954-956.indd Page 954 19/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

BIBLIOGRAPHY 955

Forsythe, G. E., M. A. Malcolm, and C. B. Moler, Computer Methods for Mathematical Computation, Prentice-Hall, Englewood Cliffs, NJ, 1977.

Fylstra, D., L. S. Lasdon, J. Watson, and A. Waren, “Design and Use of the Microsoft Excel Solver,” Interfaces, 28(5):29–55, 1998.

Gabel, R. A., and R. A. Roberts, Signals and Linear Systems, Wiley, New York, 1987.

Gear, C. W., Numerical Initial-Value Problems in Ordinary Differential Equations, Prentice-Hall, Englewood Cliffs, NJ, 1971.

Gerald, C. F., and P. O. Wheatley, Applied Numerical Analysis, 7th ed., Addison-Wesley, Reading, MA, 2004.

Gill, P. E., W. Murray, and M. H. Wright, Practical Optimization, Academic Press, London, 1981.

Gladwell, J., and R. Wait, A Survey of Numerical Methods of Partial Differential Equations, Oxford University Press, New York, 1979.

Goldberg, D. E., Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, MA, 1989.

Guest, P. G., Numerical Methods of Curve Fitting, Cambridge University Press, New York, 1961.

Hamming, R. W., Numerical Methods for Scientists and Engineers, 2d ed., McGraw-Hill, New York, 1973.

Hartley, H. O., “The Modifi ed Gauss-Newton Method for Fitting Non-linear Regression Functions by Least Squares,” Technometrics, 3:269–280, 1961.

Hayt, W. H., and J. E. Kemmerly, Engineering Circuit Analysis, McGraw-Hill, New York, 1986.

Heideman, M. T., D. H. Johnson, and C. S. Burrus, “Gauss and the History of the Fast Fourier Transform,” IEEE ASSP Mag. , 1(4):14–21, 1984.

Henrici, P. H., Elements of Numerical Analysis, Wiley, New York, 1964.

Hildebrand, F. B., Introduction to Numerical Analysis, 2d ed., McGraw-Hill, New York, 1974.

Hillier, F. S., and G. J. Lieberman, Introduction to Operations Research , 8th ed., McGraw-Hill, New York, 2005.

Hoffman, J., Numerical Methods for Engineers and Scientists, McGraw-Hill, New York, 1992.

Holland, J. H., Adaptation in Natural and Artifi cial Systems, University of Michigan Press, Ann Arbor, MI, 1975.

Hornbeck, R. W., Numerical Methods, Quantum, New York, 1975.

Householder, A. S., Principles of Numerical Analysis, McGraw-Hill, New York, 1953.

Householder, A. S., The Theory of Matrices in Numerical Analy- sis, Blaisdell, New York, 1964.

Householder, A.S. The Numerical Treatment of a Single Nonlinear Equation, McGraw-Hill, New York, 1970.

Huebner, K. H., and E. A. Thornton, The Finite Element Method for Engineers, Wiley, New York, 1982.

Hull, T. E., and A. L. Creemer, “The Effi ciency of Predictor- Corrector Procedures,” J. Assoc. Comput. Mach., 10:291, 1963.

Isaacson, E., and H. B. Keller, Analysis of Numerical Methods, Wiley, New York, 1966.

Jacobs, D. (ed.), The State of the Art in Numerical Analysis, Academic Press, London, 1977.

James, M. L., G. M. Smith, and J. C. Wolford, Applied Numerical Methods for Digital Computations with FORTRAN and CSMP, 3d ed., Harper & Row, New York, 1985.

Keller, H. B., Numerical Methods for Two-Point Boundary-Value Problems, Wiley, New York, 1968.

Lapidus, L., and G. F. Pinder, Numerical Solution of Partial Differential Equations in Science and Engineering, Wiley, New York, 1981.

Lapidus, L., and J. H. Seinfi eld, Numerical Solution of Ordinary Differential Equations, Academic Press, New York, 1971.

Lapin, L. L., Probability and Statistics for Modern Engineering, Brooks/Cole, Monterey, CA, 1983.

Lasdon, L. S., and S. Smith, “Solving Large Nonlinear Programs Using GRG,” ORSA Journal on Computing, 4(1):2–15, 1992.

Lasdon, L. S., A. Waren, A. Jain, and M. Ratner, “Design and Testing of a Generalized Reduced Gradient Code for Nonlinear Programming,” ACM Transactions on Mathematical Software, 4(1):34–50, 1978.

Lawson, C. L., and R. J. Hanson, Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ, 1974.

Luenberger, D. G., Introduction to Linear and Nonlinear Programming, Addison-Wesley, Reading, MA, 1984.

Lyness, J. M., “Notes on the Adaptive Simpson Quadrature Routine,” J. Assoc. Comput. Mach., 16:483, 1969.

Malcolm, M. A., and R. B. Simpson, “Local Versus Global Strategies for Adaptive Quadrature,” ACM Trans. Math. Software, 1:129, 1975.

Maron, M. J., Numerical Analysis, A Practical Approach, Macmillan, New York, 1982.

Milton, J. S., and J. C. Arnold, Introduction to Probability and Statistics: Principles and Applications for Engineering and the Computing Sciences, 4th ed., McGraw-Hill, New York, 2002.

Moler, C. B., Numerical Computing with MATLAB , SIAM, Philadelphia, 2005.

Muller, D. E., “A Method for Solving Algebraic Equations Using a Digital Computer,” Math. Tables Aids Comput., 10:205, 1956.

Na, T. Y., Computational Methods in Engineering Boundary Value Problems, Academic Press, New York, 1979.

Noyce, R. N., “Microelectronics,” Sci. Am., 237:62, 1977.

cha9792x_bib_954-956.indd Page 955 19/11/13 6:55 PM F-468 cha9792x_bib_954-956.indd Page 955 19/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

956 BIBLIOGRAPHY

Oppenheim, A. V., and R. Schafer, Digital Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1975.

Ortega, J., and W. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.

Ortega, J. M., Numerical Analysis— A Second Course, Academic Press, New York, 1972.

Prenter, P. M., Splices and Variational Methods, Wiley, New York, 1975.

Press, W. H., B. P. Flanner, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes: The Art of Scientifi c Computing, 3rd ed. Cambridge University Press, Cambridge, 2007.

Rabinowitz, P., “Applications of Linear Programming to Numerical Analysis,” SIAM Rev., 10:121–159, 1968.

Ralston, A., “Runge-Kutta Methods with Minimum Error Bounds,” Match. Comp., 16:431, 1962.

Ralston, A., and P. Rabinowitz, A First Course in Numerical Analysis, 2d ed., McGraw-Hill, New York, 1978.

Ramirez, R. W., The FFT, Fundamentals and Concepts, Prentice-Hall, Englewood Cliffs, NJ, 1985.

Rao, S. S., Engineering Optimization: Theory and Practice, 3d ed., Wiley-Interscience, New York, 1996.

Revelle, C. S., E. E. Whitlach, and J. R. Wright, Civil and Environmental Systems Engineering, Prentice-Hall, Englewood Cliffs, NJ, 1997.

Rice, J. R., Numerical Methods, Software and Analysis, McGraw-Hill, New York, 1983.

Ruckdeschel, F. R., BASIC Scientifi c Subroutine, Vol. 2, Byte/McGraw-Hill, Peterborough, NH, 1981.

Scarborough, J. B., Numerical Mathematical Analysis, 6th ed., Johns Hopkins Press, Baltimore, MD, 1966.

Scott, M. R., and H. A. Watts, “A Systematized Collection of Codes for Solving Two-Point Boundary-Value Problems,” in Numerical Methods for Differential Equations, L. Lapidus and W. E. Schiesser (eds.), Academic Press, New York, 1976.

Shampine, L. F., and R. C. Allen, Jr., Numerical Computing: An Introduction, Saunders, Philadelphia, 1973.

Shampine, L. F., and C. W. Gear, “A User’s View of Solving Stiff Ordinary Differential Equations,” SIAM Review, 21:1, 1979.

Simmons, E. F., Calculus with Analytical Geometry, McGraw-Hill, New York, 1985.

Stark, P. A., Introduction to Numerical Methods, Macmillan, New York, 1970.

Stasa, F. L., Applied Finite Element Analysis for Engineers, Holt, Rinehart and Winston, New York, 1985.

Stewart, G. W., Introduction to Matrix Computations, Academic Press, New York, 1973.

Swokowski, E. W., Calculus with Analytical Geometry, 2d ed., Prindle, Weber and Schmidt, Boston, 1979.

Taylor, J. R., An Introduction to Error Analysis, University Science Books, Mill Valley, CA, 1982.

Tewarson, R. P., Sparse Matrices, Academic Press, New York, 1973. Thomas, G. B., Jr., and R. L. Finney, Calculus and Analytical

Geometry, 5th ed., Addison-Wesley, Reading, MA, 1979. Van Valkenburg, M. E., Network Analysis, Prentice-Hall,

Englewood Cliffs, NJ, 1974. Varga, R., Matrix Iterative Analysis, Prentice-Hall, Englewood

Cliffs, NJ, 1962. Vichnevetsky, R., Computer Methods for Partial Differential

Equations, Vol. 1: Elliptical Equations and the Finite Element Method, Prentice-Hall, Englewood Cliffs, NJ, 1981.

Vichnevetsky, R., Computer Methods for Partial Differential Equations, Vol. 2: Initial Value Problems, Prentice-Hall, Englewood Cliffs, NJ, 1982.

Wilkinson, J. H., The Algebraic Eigenvalue Problem, Oxford University Press, Fair Lawn, NJ, 1965.

Wilkinson, J. H., and C. Reinsch, Linear Algebra: Handbook for Automatic Computation, Vol. 11, Springer-Verlag, Berlin, 1971.

Wold, S., “Spline Functions in Data Analysis,” Technometrics, 16(1):1–11, 1974.

Yakowitz, S., and F. Szidarovsky, An Introduction to Numerical Computation, Macmillan, New York, 1986.

Young, D. M., Iterative Solution of Large Linear Systems, Academic Press, New York, 1971.

Zienkiewicz, O. C., The Finite Element Method in Engineering Science, McGraw-Hill, London, 1971.

cha9792x_bib_954-956.indd Page 956 19/11/13 6:55 PM F-468 cha9792x_bib_954-956.indd Page 956 19/11/13 6:55 PM F-468 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

957

INDEX

A Absolute value, normalized, 68 Accuracy, 58–59, 112 Adams-Bashforth, 771–773 Adams-Moulton, 773–774 Adaptive quadrature, 601, 633, 640–642 Adaptive Runge-Kutta (RK) methods,

707, 744–751 Adaptive step-size control, 745, 748–749, 768 Addition, 73

large and small number, 75–76 matrix operations, 236 smearing, 77–79

Advanced methods/additional references, 113–114 curve fi tting, 584–585 linear algebraic equations, 342–343 numerical integration, 695 ordinary differential equations (ODEs),

842–843 partial differential equations (PDEs), 932 roots of equations, 227–229

Air resistance falling parachutist problem, 14–18 formulation, 14

Alternating-direction implicit (ADI) method, 850, 878–882, 885–888, 931, 932

Amplitude, 528–529 Analytical methods of problem solving

falling parachutist problem, 14–17. See also Falling parachutist problem

nature of, 14, 15 Angular frequency, 529 Antidifferentiation, 599 Approximations, 55–64. See also Estimation

accuracy/inaccuracy, 58–59, 112 approximate percent relative error,

61, 62, 114 computer algorithm for iterative

calculations, 62–64 error calculation, 59–62 error defi nitions, 59–64 fi nite-element methods, 891–894 polynomial, 83–85

precision/imprecision, 58–59 signifi cant fi gures/digits, 56–57 Taylor series, 81–86, 92–97, 655–658

Areal integrals, 596 Arithmetic mean, 444 Arithmetic operations, 73–74, 937–939,

944–945 Assemblage property matrix, 894 Associative property, matrix operations, 238 Augmentation, matrix operations, 239–240 Auxiliary conditions, 705

B Background information

blunders, 106–107 computer programming and software, 27–28 conservation laws and engineering, 18–21 curve fi tting, 443–452 data uncertainty, 58–59, 107, 662 differential calculus, 597–599 eigenvalue problems, 789 error propagation, 97–101, 114 Excel, 39–43. See also Excel formulation errors, 107 integral calculus, 597–599 linear algebraic equations, 233–241 Mathcad, 47–48, 943–953. See also

Mathcad MATLAB, 43–47, 935–942. See also

MATLAB modular programming, 37–39 numerical differentiation, 93–97 optimization, 350–351 ordinary differential equations (ODEs),

703–705 overview of problem-solving process, 12 polynomials, 176–179 root equation, 119–120, 179–182 round-off errors, 65–79, 103–105 simple mathematical model, 11–18 structured programming, 28–37 Taylor series, 81–97 total numerical error, 101–106 truncation errors, 81, 89–93, 103–105

Back substitution, 254–256 LU decomposition, 283, 301

Backward defl ation, 182 Backward difference approximation, 93–94 Bairstow’s method, 187–191, 226 Banded matrices, 300–301 Base-2 (binary) number system, 65, 67–72 Base-8 (octal) number system, 65 Base-10 (decimal) number system, 65, 66–67,

73–74 Basic feasible solution, 397, 398 Basic variables, 397 BFGS algorithm, 388, 401 Bilinear interpolation, 522–523 Binary (base-2) number system, 65, 67–72 Binding constraints, 394 Bisection method, 120, 127–135, 226, 356–357

bisection algorithm, 133, 134 computer methods, 132–133 defi ned, 127 error estimates, 129–133 false-position method versus, 137, 138–139 graphical method, 128–129, 130, 131, 228 incremental search methods versus, 127 minimizing function evaluations, 134–135 problem statement/solution, 128–129 termination criteria, 129

Blunders, 106–107 Boole’s rule, 622, 623, 641–642 Boundary conditions

derivative, 787–788, 860–863, 877 fi nite-element methods, 894, 903,

906–908, 927 irregular boundaries, 863–866 Laplace equation, 850, 855–858, 860–866

Boundary-value problems, 705, 782–789, 842 eigenvalue, 792–795 shooting method, 707, 782, 783–786

Bracketing methods, 123–141, 227, 366 bisection method, 120, 127–135, 226,

356–357 computer methods, 126–127 defi ned, 123 false-position method, 120, 135–141, 226

cha9792x_ndx_957-976.indd Page 957 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 957 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

958 INDEX

Bracketing methods—Cont. graphical method, 123–127, 128 incremental searches/determining initial

guesses, 141 Break command, 46 Break loops, 32, 33–34 Brent’s method, 120–121, 162–166, 226, 227

computer methods, 164–166, 366–368 graphical method, 162, 163 inverse quadratic interpolation, 162–164 optimization, 352, 356, 366–368, 438 root of polynomials, 199

Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithms, 388, 401

B splines, 585 Butcher’s fi fth-order Runge-Kutta method,

737–739 Butterfl y network, 547, 549

C C11, 48 Cartesian coordinates, 596 CASE structure, 31, 32, 41, 45 Cash-Karp RK method, 747, 749–750 Centered fi nite divided-difference

approximation, 94, 95 Central Limit Theorem, 449 Chaotic solutions, 822 Characteristic, 67–68 Characteristic equation, 177–178 Charge, conservation of, 20 Chebyshev economization, 585 Chemical/biological engineering

analyzing transient response of reactor, 811–818

conservation of mass, 20 determining total quantity of heat, 673–675 ideal gas law, 204–207 integral calculus, 673–675 least-cost design of a tank, 416–420 linear algebraic equations, 319–322 linear regression, 563–567 one-dimensional mass balance of reactor,

915–919 optimization, 416–420 ordinary differential equations (ODEs),

811–818 partial differential equations (PDEs),

915–919 population growth models, 563–567 roots of equations, 204–207 steady-state analysis of system of reactors,

319–322

Cholesky decomposition, 302–304 Chopping, 70–71 Civil/environmental engineering

analysis of statically determinate truss, 322–326

conservation of momentum, 20 curve fi tting, 567–568 defl ections of a plate, 919–921 effective force on mast of racing sailboat,

675–677 greenhouse gases and rainwater, 207–209 integral calculus, 675–677 least-cost treatment of wastewater,

421–424 linear algebraic equations, 322–326 optimization, 416, 421–424 ordinary differential equations (ODEs),

818–822 partial differential equations (PDEs),

919–921 predator-prey models and chaos,

818–822 roots of equations, 207–209 splines to estimate heat transfer, 567–568

Classical fourth-order Runge-Kutta method, 735–737, 843

Coeffi cient, method of undetermined, 644–645 Coeffi cient of determination, 463 Coeffi cient of interpolating polynomial, 507 Coeffi cient of thermal conductivity, 853 Coeffi cient of variation, 445 Colebrook equation, 212, 214 Column vectors, 234 Commutative property, matrix operations,

236, 238 Complex systems, linear algebraic

equations, 271 Composite, integration formulas, 609–612 Computational error, 59–62, 74 Computer programming and software, 27–48, 111.

See also Pseudocode algorithms bisection method, 133, 134 bracketing methods, 126–127 Brent’s method, 164–166 computer programs, defi ned, 28 cost comparison, 111–113 curve fi tting, 449–450, 454, 465–468,

475–476, 478–479, 552–560 Excel. See Excel linear algebraic equations, 243–244,

269–270, 311–316 linear programming, 402–404 linear regression, 464–468

Mathcad. See Mathcad MATLAB. See MATLAB modular programming, 37–39 numerical integration/differentiation,

663–670 optimization, 352, 402–413, 419–420 ordinary differential equations (ODEs), 707,

718–721, 728, 739, 742–744, 749–751, 801–808

other languages and libraries, 48 partial differential equations (PDEs), 850,

869–870, 908–912 roots of equations, 126–127, 192–201 software user types, 27–28 step-size control, 768 structured programming, 28–37 trapezoidal rule algorithms, 612–615

Condition numbers, 100–101 matrix, 241, 294–296

Confi dence intervals, 446–452, 481–482 Conjugate directions, 374 Conjugate gradient, 352, 386–388, 401 Conservation laws, 18–21

by fi eld of engineering, 20 simple models in specifi c fi elds, 19, 20 stimulus-response computations, 290–291

Conservation of charge, 20 Conservation of energy, 20 Conservation of mass, 20, 319–322 Conservation of momentum, 20 Constant of integration, 704 Constant step size, 768 Constitutive equation, 853–854 Constrained optimization, 351, 352, 390–401

linear programming, 350, 352, 390–401 nonlinear, 352, 401, 404–408, 413

Constraints binding/nonbinding, 394 optimization, 348, 350

Continuous Fourier series, 533–536 approximation, 534–536 determination of coeffi cients, 534

Control-volume approach, 866–869 Convergences

defi ned, 877 fi xed-point iteration, 147–150 Gauss-Seidel (Liebmann) method, 306–309 linear, 147–150 nature of, 150 of numerical methods of problem

solving, 111 Cooley-Tukey algorithm, 545, 550–551 Corrector equation, 723, 759–760

cha9792x_ndx_957-976.indd Page 958 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 958 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 959

Correlation coeffi cient, 463 Count-controlled loops, 33–34, 41, 45, 46 Cramer’s Rule, 248, 249–250, 341 Crank-Nicolson method, 850, 882–885,

931, 932 Critically damped case, 178–179 Crout decomposition, 285–287 Cubic splines, 511, 517–521, 582, 584, 695

computer algorithms, 520–521 derivation, 518–519 interpolation with Mathcad, 559–560

Cumulative normal distribution, 651–653 Current balance, 20 Curvature, 588 Curve fi tting, 441–585

advanced methods and additional references, 584–585

case studies, 563–571 coeffi cients of an interpolating

polynomial, 507 comparisons of alternative methods,

582–583 computer methods, 449–450, 454, 465–468,

475–476, 478–479, 552–560 defi ned, 441 engineering applications, 442–443, 563–571 estimation of confi dence intervals, 446–452,

481–482 extrapolation, 508–509 Fourier approximation, 479, 526–560, 583 general linear least squares model,

453, 479–483 goals/objectives, 454–455 important relationships and formulas,

583–584 interpolation, 441, 454, 490–509 inverse interpolation, 507–508 Lagrange interpolating polynomial, 454,

490, 502–507, 509, 582, 584 least-squares regression, 441, 452, 456–486 linear regression, 452, 456–472 mathematical background, 443–452 multidimensional interpolation, 521–523 multiple linear regression, 452, 476–479,

582, 583, 584 Newton’s divided-difference interpolating

polynomials, 491–502 Newton’s interpolating polynomial, 454,

490, 491–502, 504–505, 509, 582, 584 noncomputer methods, 441–442 nonlinear regression, 470, 483–486,

555, 582 normal distribution, 446

polynomial regression, 452, 472–476, 583, 584 power spectrum, 551–552 scope/preview, 452–454 simple statistics, 443–446 with sinusoidal functions, 527–533 spline interpolation, 454, 511–521 time domains, 536–540

D Data distribution, 446 Data uncertainty, 58–59, 107, 662 Davidon-Fletcher-Powell (DFP) method of

optimization, 388, 439 Decimal (base-10) number system, 65, 66–67,

73–74 Decimation-in-frequency, 545 Decimation-on-time, 545 Decision loops, 32 Defi nite integration, 588n Defl ation, 800

forward, 181–182 polynomial, 180–182

Degrees of freedom, 444 Dependent variables, 11–12, 118, 699 Derivative boundary conditions, 787–788,

860–863, 877 Derivative mean-value theorem, 88 Descriptive models, 346 Design, 21 Design variables, 35, 350 Determinants, in Gauss elimination, 248–249,

261–263 Determination, coeffi cient of, 463 Diagonally dominant systems, 309 Differential calculus, 587–590, 655–670. See

also Numerical differentiation; Optimization; Ordinary differential equations (ODEs); Partial differential equations (PDEs) data with errors, 661–662 differentiate, defi ned, 587–588 differentiation of unequally spaced data,

660–661 differentiation with computer software,

663–670 engineering applications, 593–594, 663–670 fi rst derivative, 587–588, 656–657 goals/objectives, 601–602 high-accuracy differentiation formulas,

601, 655–658 mathematical background, 597–599 noncomputer methods for differentiation,

590–592

numerical differentiation with software packages, 663–670

partial derivatives, 588, 662–663 Richardson’s extrapolation, 601, 633,

635–638, 641, 658–660 scope/preview, 599–601 second derivative, 588, 656–657 terminology, 587–589

Differential equations, 13–16, 27, 38, 699 Direct approach

fi nite-element methods, 897–901 optimization, 352, 370, 371–375

Directional derivative, 376 Dirichlet boundary condition, 787–788,

855–858, 909 Discrete Fourier transform (DFT), 542–544 Discretization, fi nite-element methods, 891,

896, 904, 924–925 Discriminant, 178 Distributed-parameter system, 916 Distributed variable systems, 232, 233 Distributive property, matrix operations, 238 Division, 74

synthetic, 180–181 by zero, 258

DOEXIT construct, 32, 33, 35, 41, 45 DOFOR loops, 33–34 Double integrals, 627–629 Double roots, 166, 167 Drag coeffi cient, 14 Dynamic instability, 919

E Eigenvalue problems, 789–808

boundary-value problem, 792–795 computer methods, 801–808 eigenvalue, defi ned, 789 eigenvalue analysis of axially loaded

column, 794–795 eigenvectors, 789, 791–792 mass-spring system, 791–792 mathematical background, 789 other methods, 800–801 physical background, 790–792 polynomial method, 177–178, 795–797 power method, 707, 797–800

Eigenvectors, 789, 791–792 Electrical engineering

conservation of charge, 20 conservation of energy, 20 currents and voltages in resistor circuits,

326–328 curve fi tting, 569–570

cha9792x_ndx_957-976.indd Page 959 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 959 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

960 INDEX

Electrical engineering—Cont. design of electrical circuit, 209–212 Fourier analysis, 569–570 integral calculus, 677–680 linear algebraic equations, 326–328 maximum power transfer for a circuit,

425–429 optimization, 416, 425–429 ordinary differential equations (ODEs),

822–827 partial differential equations (PDEs),

921–924 root-mean-square current, 677–680 roots of equations, 209–212 simulating transient current for electric

circuit, 822–827 two-dimensional electrostatic fi eld

problems, 921–924 Element properties, fi nite-element methods, 894 Element stiffness matrix, 894, 926 Elimination of unknowns, 250–256

back substitution, 254–256 forward, 252–254

Elliptic partial differential equations (PDEs), 846–847, 852–870, 931, 932 boundary conditions, 850, 860–866 computer software solutions, 869–870 control-volume approach, 866–869 Gauss-Seidel (Liebmann) method,

850, 856–858, 881 Laplace equation, 846, 850, 852–866,

922–924 Embedded Runge-Kutta (RK) method, 747 ENDDO statement, 33–34 End statement, 46 Energy

conservation of, 20 equilibrium and minimum potential, 429–430

Energy balance, 118 Engineering problem solving

chemical engineering. See Chemical/ biological engineering

civil engineering. See Civil/environmental engineering

conservation laws, 18–21 curve fi tting, 442–443, 563–571 dependent variables, 11–12, 118 differential calculus, 593–594, 663–670 electrical engineering. See Electrical

engineering falling parachutist problem. See Falling

parachutist problem

forcing functions, 11–12 fundamental principles, 118 independent variables, 11–12, 118 integral calculus, 594–597, 663–670,

673–683 linear algebraic equations, 232–233,

319–330 mechanical engineering. See Mechanical/

aerospace engineering Newton’s second law of motion, 11–18,

55, 118, 328, 702 numerical differentiation, 593–597 optimization, 346–350, 352, 416–430 ordinary differential equations (ODEs),

701–702, 707, 811–830 parameters, 11–12, 118, 816 partial differential equations (PDEs),

846–848, 850, 915–927 practical issues, 21 roots of equations, 118–119, 122, 176–179,

204–215 two-pronged approach, 11, 12, 14–18

Entering variables, 398–399 Epilimnion, 567 Equal-area graphical differentiation, 590–591 Equality constraint optimization, 350 Error(s)

approximations. See Approximations bisection method, 129–133 blunders, 106–107 calculation, 59–62, 74 data uncertainty, 58–59, 107, 662 defi ned, 55 differential calculus, 661–662 estimates for iterative methods, 61–62 estimates in multistep method, 764–765 estimation, 464 estimation for Euler’s method, 714–715 falling parachutist problem, 55 formulation, 107 Gauss quadrature, 649–650 integral calculus, 661–662 linear algebraic equations, 291–297 Newton-Raphson estimation method,

152–154 Newton’s divided-difference interpolating

polynomial estimation, 497–502 numerical differentiation, 102–105 predictor-corrector approach, 723–724,

762–767 quantizing, 70–71, 72, 75 relative, 100

residual, 457, 461–464 round-off. See Round-off errors Simpson’s 1/3 rule estimation, 616 true fractional relative error, 59 truncation. See Truncation errors

Error defi nitions, 59–64 approximate percent relative error, 61, 62, 114 stopping criterion, 62, 63, 114 true error, 59, 98, 104, 114 true percent relative error, 59, 61, 64, 114

Error propagation, 97–101, 114 condition, 100–101 functions of more than one variable, 99–100 functions of single variable, 97–98 stability, 100–101

Estimated mean, 449 Estimation. See also Approximations

confi dence interval, 446–452, 481–482 errors, 464, 497–502, 616, 714–715 Newton-Raphson estimation method,

152–154 parameter, 816 standard error of the estimate, 462 standard normal estimate, 448–449

Euler-Cauchy method. See Euler’s method Euler’s method, 16–17, 27, 38, 48, 178–179

algorithm for, 718–721 backward/implicit, 757 effect of reduced step size, 715–717 error analysis, 712–717 Euler’s formula, 794 improvements, 721–729 ordinary differential equations (ODEs),

705–707, 710–729, 841, 842, 843 as predictor, 759–760 problem statement/solution, 710–712 systems of equations, 740

Excel, 27–28, 33, 39–43 computer implementation of iterative

calculation, 63–64 curve fi tting, 552–555, 566–567 Data Analysis Toolpack, 552, 553–555 described, 39 Goal Seek, 192, 193 linear algebraic equations, 311–312 linear programming, 402–404 linear regression, 464 nonlinear constrained optimization, 404–408 optimization, 352, 402–408, 419–420,

423–425 ordinary differential equations (ODEs), 801,

817–818

cha9792x_ndx_957-976.indd Page 960 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 960 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 961

partial differential equations (PDEs), 908–910

roots of equations, 77, 192–195, 212–213 Solver, 192, 193–195, 402–408, 423–429,

566–567, 801, 817–818 standard use, 39–40 Trendline command, 552–553 VBA macros, 40–43

Explicit solution technique defi ned, 118 ordinary differential equations (ODEs),

757–759 parabolic partial differential equations

(PDEs), 874–878, 879, 885, 931 Exponent, 67–68 Exponential model of linear regression,

469–470 Extended midpoint rule, 651 Extrapolation, 508–509 Extreme points, 395 Extremium, 356–359

F Factors, polynomial, 180 Falling parachutist problem, 14–18, 118–119

analytical problem statement/solution, 14–17 computer algorithm, 269–270 error, 55 Gauss elimination, 269–270 Gauss quadrature application, 649 numerical problem statement/solution,

17–18 optimization of parachute drop cost,

347–350, 404–408 schematic diagram, 13 velocity of the parachutist, 465–468,

505–507, 709–710 False-position method, 120, 135–141, 226

bisection method versus, 137, 138–139 false-position formula, 136–138 graphical method, 135, 139, 228 modifi ed false positions, 140–141, 226 pitfalls, 138–141 problem statement/solution, 136–138 secant method versus, 158–160

Faraday’s law, 702 Fast Fourier transform (FFT), 454,

544–551, 560 Cooley-Tukey algorithm, 545, 550–551 Sande-Tukey algorithm, 545, 546–550

Feasible extreme points, 395 Feasible solution space, 392–395

Fibonacci numbers, 358–359 Fick’s law of diffusion, 702 Finish, 33–34 Finite-difference methods, 16–17, 81, 93,

95–97, 655–658 elliptic partial differential equations (PDEs),

846–847, 850, 852–870, 931, 932 high-accuracy differentiation formulas,

601, 655–658 optimization, 380–381 ordinary differential equations (ODEs),

707, 786–789 parabolic partial differential equations

(PDEs), 846, 847–848, 850, 852–870, 873–888, 931, 932

Finite-divided-difference approximations of derivatives, 93, 95–97

Finite-element methods, 890–908 assembly, 894, 901–903, 926–927 boundary conditions, 894, 903, 906–908, 927 defi ned, 890–891 discretization, 891, 896, 904, 924–925 element equations, 891–894, 897–901,

904–906, 925–926 general approach, 891–894 partial differential equations (PDEs), 850,

890–908, 931 single dimension, 895–904 solution and postprocessing, 894, 904, 908, 927 two dimensions, 904–908

First backward difference, 93–94 First derivative, 587–588, 656–657 First fi nite divided difference, 93 First forward difference, 93, 114 First forward fi nite divided difference, 93, 95,

96, 114 First-order approximation, 81–82, 84, 86, 92–93 First-order methods, 699, 715 First-order splines, 513–514 Fixed (Dirichlet) boundary condition, 787–788,

855–858, 909 Fixed-point iteration, 146–151, 226

algorithm, 150–151 convergences, 147–150 graphical method, 147–150 nonlinear equations, 170–171

Fletcher-Reeves conjugate gradient algorithm, 386–387, 439

Floating-point operations/fl ops, 256–258 Floating-point representation, 67–72

chopping, 70–71 fractional part/mantissa/signifi cand, 67–68

integer part/exponent/characteristic, 67–68 machine epsilon, 71–72 overfl ow error, 69–70 quantizing errors, 70–71, 72, 75

Flowcharts, 29–31 sequence structure, 30 simple selection constructs, 31 symbols, 29

Force balance, 20, 118 Forcing functions, 11–12 Formulation errors, 107 Fortran 90, 48, 74–75 Forward defl ation, 181–182 Forward elimination of unknowns, 252–254 Forward substitution, LU decomposition,

283, 301 Fourier approximation, 479, 526–552, 583

continuous Fourier series, 534–536 curve fi tting with sinusoidal functions,

527–533 defi ned, 526–527 discrete Fourier transform (DFT), 542–544 engineering applications, 569–570 fast Fourier transform (FFT), 454,

544–551, 560 Fourier integral and transform, 540–551 frequency domain, 536–540 power spectrum, 551–552 time domain, 536–540

Fourier integral, 540–542 Fourier series, 533–536, 537, 933–934 Fourier’s law of heat conduction, 593–594, 702,

853, 921 Fourier transform, 540–551

discrete Fourier transform (DFT), 542–544

fast Fourier transform (FFT), 454, 544–551, 560

Fourier transform pair, 540 Fourth derivative, 656–657 Fourth-order methods

Adams, 771–774, 777–779, 841, 842, 843 Runge-Kutta, 735–737, 741–742, 743, 746,

750–751, 843 Fractional parts, 67–68 Fractions, fl oating-point representation, 68 Frequency domain, 536–540 Frequency plane, 536–537 Friction factor, 212 Frobenius norm, 294 Fully augmented version, 396 FUNCTION, 38

cha9792x_ndx_957-976.indd Page 961 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 961 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

962 INDEX

Function(s) error propagation, 97–100 forcing, 11–12 interpolation, 892–893 mathematical behavior, 112 modular programming, 37 penalty, 401 sinusoidal, 527–533 spline, 511, 585

Functional approximation, 585 Fundamental frequency, 533 Fundamental theorem of integral calculus, 598

G Gauss elimination, 245–275, 343

computer algorithm, 269–270 Cramer’s Rule, 248, 249–250, 341 determinants, 248–249, 261–263 elimination of unknowns, 250–251 Gauss-Jordan method, 273–275 graphical method, 245–247 improving solutions, 264–269 LU decomposition version, 280–285 more signifi cant fi gures, 264 naive, 252–258 operation counting, 256–258 pitfalls of elimination methods, 258–264 pivoting, 241, 253–254, 258, 264–269, 341 solving small numbers of equations, 245–251

Gauss-Jordan method, 273–275 Gauss-Legendre formulas, 643, 645–650,

679, 695 higher-point, 648–649 two-point, 645–648

Gauss-Newton method, 483–486, 585 Gauss quadrature, 601, 633, 642–650, 680,

694, 696 error analysis, 649–650 Gauss-Legendre formulas, 643, 645–650,

679, 695 method of undetermined coeffi cients,

644–645 Gauss-Seidel (Liebmann) method, 241–242,

300, 304–311, 341–342, 343, 931 algorithm, 309–310 convergence criterion, 306–309 elliptic partial differential equations (PDEs),

850, 856–858, 881 graphical method, 307 iteration cobwebs, 308 problem contexts, 310–311 relaxation, 309

Generalized reduced gradient (GRG), 401, 439 General linear least-squares model, 453, 479–483

confi dence intervals for linear regression, 481–482

general matrix formulation, 479–480 statistical aspects of least-squares theory,

480–483 General solution, 177 Genetic algorithm, 373 Given’s method, 801 Global truncation error, 713 Golden ratio, 358–359 Golden-section search optimization, 352,

356–363, 427–428, 438 extremium, 356–359 golden ratio, 358–359 single-variable optimization, 356 unimodal, 356–357

Gradient, defi ned, 594 Gradient methods of optimization, 352, 370,

375–388 conjugate gradient method (Fletcher-Reeves),

352, 386–387, 439 fi nite difference approximation, 380–381 gradients, 376–378 Hessian, 352, 378–380, 439 Marquardt’s method, 352, 387–388, 585 path of steepest ascent/descent, 352,

377–378, 381–386, 585 quasi-Newton methods, 352, 388, 401, 439

Graphical methods bisection, 128–129, 130, 131, 228 bracketing, 123–127, 128 Brent’s method, 162, 163 false-position method, 135, 139, 228 fi xed-point iteration, 147–150 Gauss elimination, 245–247 Gauss-Seidel (Liebmann) method, 307 linear algebraic equations, 245–247,

320–321, 323, 325, 327–329, 341 linear programming, 392–395 Newton-Raphson method, 151, 156, 228 open, 145 roots of equations, 117, 120–121, 123–127,

145, 146–151, 157–166, 226 secant, 157, 159, 160, 162, 228

Greenhouse gases, 207–209

H Half-saturation constant, 563–564 Hamming’s method, 779 Harmonics, 533

Hazen-Williams-equation, 571 Heat balance, 118 Heat-conduction equation, 846, 847–848,

873–888. See also Parabolic partial differential equations (PDEs)

Hessenberg form, 801 Hessian, 352, 378–380, 439 Heun’s method, 707, 722–726, 728, 729, 732,

841, 843 High-accuracy differentiation formulas, 601,

655–658 Histograms, 447–448 Hooke’s law, 328, 429–430 Hotelling’s method, 800 Householder’s method, 801 Hyperbolic partial differential equations (PDEs),

846, 848 Hypolimnion, 567 Hypothesis testing, 442–443

I Ideal gas law, 204–207 IEEE format, 72 IF/THEN/ELSE/IF structure, 31, 41, 45 IF/THEN/ELSE structure, 30, 31, 32, 35, 41, 45 IF/THEN structure, 30, 31, 37, 41, 45, 266 Ill-conditioned systems, 101, 259–263

effect of scale on determinant, 261–263 elements of matrix inverse as measure

of, 292 singular systems, 247, 263–264

Implicit solution technique defi ned, 119 ordinary differential equations (ODEs),

707, 755, 757–759 parabolic partial differential equations (PDEs),

850, 878–882, 885–888, 931, 932 Imprecision, 58–59 Improper integrals, 601, 633, 650–653

cumulative normal distribution, 651–653 extended midpoint rule, 651 normalized standard deviate, 651–653

Improved polygon method. See Midpoint (improved polygon) method

Inaccuracy, 58–59 Incremental search methods

bisection method versus, 127 defi ned, 127 determining initial guesses, 141

Increment function, 729–730 Indefi nite integral, 700 Indefi nite integration, 588n

cha9792x_ndx_957-976.indd Page 962 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 962 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 963

Independent variables, 11–12, 118, 699 Indexes, 33–34 Inequality constraint optimization, 350 Inferential statistics, 447, 449 Initial value, 705 Initial-value problems, 705, 781 Inner products, 79 In place implementation, 266 INPUT statements, 38 Integer part, 67–68 Integer representation, 65–67 Integral, 348–349 Integral calculus, 588–590, 633–653

Adams formula, 771–774, 777–779, 841, 842, 843

adaptive quadrature, 601, 633, 640–642 Boole’s rule, 622, 623, 641–642 calculation of integrals, 594–597 closed forms, 599–601, 604–605, 622–624,

633–634, 771, 773–774 data with errors, 661–662 engineering applications, 594–597,

663–670, 673–683 fundamental theorem, 598 Gauss quadrature, 601, 633, 642–650, 680,

694, 696 goals/objectives, 601–602 improper integrals, 601, 633, 650–653 integrate, defi ned, 588 integration with computer software, 663–670 integration with unequal segments,

601, 624–627 mathematical background, 597–599 multiple integrals, 627–629 Newton-Cotes formulas, 599–601, 603–629,

633–634, 694–695, 728, 768–771 noncomputer methods for integration, 592 numerical integration with software

packages, 663–670 open forms, 601, 604–605, 627, 633,

650–651, 770–773 Richardson’s extrapolation, 601, 633,

635–638, 641, 658–660 Romberg integration, 601, 633, 634–640,

641, 679, 694, 696 scope/preview, 599–601 Simpson’s 1/3 rule, 600, 615–620, 622, 623,

676, 694, 696 Simpson’s 3/8 rule, 600, 620–622, 623,

694, 696 Simpson’s rules, 600, 615–624, 625–627,

641, 696

terminology, 588 trapezoidal rule, 600, 605–615, 623,

624–627, 636–637, 676, 694, 696, 726 Integral form, 82 Integrand, 588, 664 Interdependent computations, 74–75 Interpolation

coeffi cients of interpolating polynomial, 507 computers in, 505–507, 559–560 curve fi tting, 441, 454, 490–509 with equally spaced data, 508–510 fi nite-element methods, 892–893 interpolation functions, 892–893 inverse, 507–508 inverse quadratic, 162–164 Lagrange interpolating polynomials, 454,

490, 502–507, 509, 582, 584 linear interpolation method, 162, 491–492 multidimensional, 521–523 Newton’s divided-difference interpolating

polynomials, 454, 490, 491–502, 504–505, 509, 582, 584

polynomial, 490–509 quadratic, 493–495 spline, 454, 511–521

Interval estimator, 447 Inverse Fourier transform of F, 540–541 Inverse interpolation, 507–508 Inverse quadratic interpolation, 162–164 Irregular boundaries, 863–866 Iterative approach to computation

computer algorithms, 62–64 defi ned, 60–61 error estimates, 61–62 Gauss-Seidel (Liebmann) method, 241–242,

300, 304–311, 341–342, 343, 931 iterative refi nement, 296–297

J Jacobian, 172 Jacobi iteration, 306 Jacobi’s method, 800–801 Jenkins-Traub method, 192, 229

K Kirchhoff’s laws, 118, 209–212, 326, 822

L Lagging phase angle, 529 Lagrange interpolating polynomials, 454, 490,

502–507, 509, 582, 584 Lagrange multiplier, 346, 424

Lagrange polynomial, 163 Laguerre’s method, 192, 199, 229 Laplace equation, 846, 852–866, 922–924

boundary conditions, 850, 855–858, 860–866 described, 852–854 fl ux distribution of heated plate, 859–860 Liebmann method, 850, 856–858, 931 secondary variables, 859–860 solution technique, 854–860

Laplacian difference equation, 855–856 Large computations, interdependent

computations, 74–75 Large versus small systems, 21 Least-squares fi t of a sinusoid, 530–533 Least-squares regression

curve fi tting, 441, 452, 456–486 general linear least-squares model, 453,

479–483 least-squares fi t of a straight line, 459–461 linear regression, 452, 456–472, 582–584

Leaving variables, 398–399 Levenberg-Marquardt method, 412 Liebmann method. See Gauss-Seidel

(Liebmann) method Linear algebraic equations, 231–343

advanced methods and additional references, 342–343

case studies, 319–330 comparisons of methods, 341–342 complex systems, 271 computer methods, 243–244, 269–270,

311–316 Cramer’s rule, 248, 249–250, 341 determinants, 248–249 distributed variable systems, 232, 233 division by zero, 258 elimination of unknowns, 250–251 engineering applications, 232–233, 319–330 error analysis, 291–297 Gauss elimination, 241, 245–277, 341, 343 Gauss-Jordan method, 273–275 Gauss-Seidel (Liebmann) method, 241–242,

300, 304–311, 341–342, 343, 931 general form, 231 goals/objectives, 243–244 graphical method, 245–247, 320–321, 323,

325, 327–329, 341 ill-conditioned systems, 247, 259–263 important relationships and formulas,

342, 343 LU decomposition methods, 241, 278–287,

330, 341, 343

cha9792x_ndx_957-976.indd Page 963 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 963 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

964 INDEX

Linear algebraic equations—Cont. lumped variable systems, 232, 233 mathematical background, 233–241 matrix inverse, 238–239, 241, 287–291 matrix notation, 234–235 matrix operating rules, 236–240 more signifi cant fi gures, 264 noncomputer methods for solving, 231–232 nonlinear systems of equations, 271–273 pivoting, 241, 253–254, 258, 264–266 representing in matrix form, 240–241 round-off errors, 259 scaling, 261–263, 266–269 scope/preview, 241–243 singular systems, 247, 263–264 special matrices, 300–304 system condition, 291–297

Linear convergences, 147–150 Linear interpolation method. See also Brent’s

method; False-position method defi ned, 162, 491–492 linear-interpolation formula, 491–492

Linearization, 700–701 Linear programming

computer solutions, 402–404 defi ned, 390 feasible solution space, 392–395 graphical solution, 392–395 optimization, 346, 350, 352, 390–401 possible outcomes, 394–395 setting up problem, 391–392 simplex method, 346, 352, 396–401 standard form, 390–392

Linear regression, 456–472, 582–584 computer programs, 464–468 confi dence intervals, 481–482 criteria for “best” fi t, 458–459 curve fi tting, 452, 456–472 estimation errors, 464 exponential model, 469–470 general comments, 472 general linear least-squares model,

479–483 least-squares fi t of straight line, 459–461 linearization of nonlinear relationships,

468–472 linearization of power equation, 470–472 minimax criterion, 459, 585 multiple, 452, 476–479, 582, 583, 584 quantifi cation of error, 461–464 residual error, 457, 461–464 standard error of the estimate, 462

Linear splines, 511–514 Linear trend, 82–83 Line spectra, 538–540 Local truncation error, 713 Logical representation, 30–37

algorithm for root of a quadratic, 34–37 repetition, 31–34 selection, 30–31 sequence, 30

Lorenz equations, 819–822 Lotka-Volterra equations, 818–822 LR method (Rutishauser), 801 LU decomposition methods, 241, 278–287,

330, 341, 343 algorithm, 282, 284–285, 286–287 Crout decomposition, 285–287 defi ned, 278 LU decomposition step, 279, 280, 283, 284,

300, 301 overview, 279–280 substitution step, 279, 280, 283–285 version of Gauss elimination, 280–285

Lumped-parameter systems, 915–916 Lumped variable systems, 232, 233

M MacCormack’s method, 877–878 Machine epsilon, 71–72 Maclaurin series expansion, 61–62 Mantissa, 67–69 Maple V, 48 Marquardt’s method, 352, 387–388, 585 Mass, conservation of, 20, 319–322 Mass balance, 20, 118 Mathcad, 47–48, 943–953

basics, 943–944 curve fi tting, 558–560 entering text, 944 graphics, 949–951 linear algebraic equations, 314–316 mathematical functions and variables,

945–948 mathematical operations, 944–945 Minerr, 412 multigrid function, 911–912 multiline procedures/subprograms, 949 numerical integration/differentiation, 669–670 numerical methods function, 948 online help, 953 optimization, 352, 412–413 ordinary differential equations (ODEs),

806–808

partial differential equations (PDEs), 911–912

QuickSheets, 953 relax function, 911–912 resource center, 953 roots of equations, 199–201, 212–213 symbolic mathematics, 951–953 ToolTips, 953

Mathematical laws, 20 Mathematical models, defi ned, 11–12 Mathematical programming. See Optimization Mathsoft Inc., 47 MathWorks, Inc., 43 MATLAB, 27–28, 33, 935–942

assignment of values to variable names, 936–937

built-in functions, 939 computer implementation of iterative

calculation, 63–64 curve fi tting, 555–558 described, 43 graphics, 939–940 linear algebraic equations, 312–314 linear regression, 464 mathematical operations, 937–939 matrix analysis, 313 M-fi les, 43–47, 103–105, 430 numerical differentiation, 103–105 numerical integration/differentiation,

663–669 optimization, 352, 366–368, 408–412, 430 ordinary differential equations (ODEs),

802–806, 815–817, 825–827 partial differential equations (PDEs),

910–911 polynomials, 940 roots of equations, 195–198, 212–213 statistical analysis, 940–941

Matrix condition number, 241, 294–296 Matrix inverse, 238–239, 241, 287–291

calculating, 288–290 stimulus-response computations, 290–291

Matrix norms, 292–294 Matrix operations

banded matrices, 300–301 Cholesky decomposition, 302–304 components, 234–235 error analysis and system condition,

291–297 matrix, defi ned, 234 matrix condition number, 241, 294–296 matrix inverse, 238–239, 241, 287–291

cha9792x_ndx_957-976.indd Page 964 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 964 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 965

matrix notation, 234–235 representing linear algebraic equations in

matrix form, 240–241 rules, 236–240 symmetric matrices, 300 tridiagonal systems, 301–302

Maximum attainable growth, 563–564 Maximum likelihood principle, 461–462 Mean value, 444, 449, 528–529

confi dence interval on the mean, 451–452 derivative mean-value theorem, 88 determining mean of discrete points,

594–595 spread around, 462

Mechanical/aerospace engineering analysis of experimental data, 570–571 conservation of momentum, 20 curve fi tting, 570–571 equilibrium and minimum potential energy,

429–430 fi nite-element solution of series of springs,

924–927 integral calculus, 680–683 linear algebraic equations, 328–330 numerical integration to compute work,

680–683 optimization, 416, 429–430 ordinary differential equations (ODEs),

827–830 partial differential equations (PDEs),

924–927 pipe friction, 212–215 roots of equations, 212–215 spring-mass systems, 328–330 swinging pendulum, 827–830

Method of false position. See False-position method

Method of lines, 877–878 Method of undetermined coeffi cients, 644–645 Method of weighted residuals (MWR),

fi nite-element methods, 897–901 M-fi les (MATLAB), 43–47, 103–105, 430 Microsoft, Inc., 39 Midpoint (improved polygon) method, 707,

726–728, 729, 733–734 Milne’s method, 775–779, 841 Minimax criterion, 459, 585 MINPACK algorithms, 412 Mixed partial derivatives, 663 Modifi ed Euler. See Midpoint (improved

polygon) method Modifi ed secant method, 161–162, 226

Modular programming, 37–39 advantages, 38 defi ned, 37

Momentum, conservation of, 20 m surplus variables, 396–397 Müller’s method, 183–187, 199, 226 Multidimensional interpolation, 521–523 Multidimensional unconstrained optimization,

370–388 direct methods (nongradient), 352, 370,

371–375 gradient methods (descent/ascent), 352, 370,

375–388 MATLAB, 410–412 pattern directions, 374–375 Powell’s method, 374–375, 386, 438 random search method, 352, 371–373 univariate search method, 352, 373

Multimodal optimization, 355–356 Multiple-application trapezoidal rule,

609–612, 696 Multiple integrals, 627–629 Multiple linear regression, 452, 476–479, 582,

583, 584 Multiple roots, 125, 166–169

double roots, 166, 167 modifi ed Newton-Raphson method for

multiple roots, 167–169, 226 Newton-Raphson method, 166–167 secant method, 166–167 triple roots, 166, 167

Multiplication, 74 inner products, 79 matrix operations, 236–238

Multistep methods, 707, 755, 759–779, 841

N Naive Gauss elimination, 252–258

back substitution, 254–256 forward elimination of unknowns, 252–254 operation counting, 256–258

n-dimensional vector, 350 Newmann boundary condition, 787–788,

860–863 Newton-Cotes integration formulas, 599–601,

603–629, 633–634, 728 Boole’s rule, 622, 623, 641–642 closed formulas, 771, 773–774 comparisons, 694–695 defi ned, 603 higher-order, 622–624, 637–638, 694 open formulas, 770–771

ordinary differential equations (ODEs), 768–771

Simpson’s 1/3 rule, 600, 615–620, 622, 623, 625–627, 676, 694, 696

Simpson’s 3/8 rule, 600, 620–622, 623, 625–627, 694, 696

trapezoidal rule, 600, 605–615, 623–627, 636–637, 676, 694, 696, 726

Newton-Raphson method, 120–121, 151–157, 206–207, 214, 227, 365 algorithm, 155–157 error estimates, 152–154 graphical method, 151, 156, 228 modifi ed method for multiple roots,

167–169, 226 multiple roots, 166–167 Newton-Raphson formula, 152 nonlinear equations, 171–173 pitfalls, 154–155 polynomials, 183, 188 slowly converging function, 154–155 Taylor series expansion, 272 termination criteria, 152–154

Newton’s divided-difference interpolating polynomials, 454, 490, 491–502, 509, 582, 584 computer algorithm, 499–502 defi ned, 496–497 derivation of Lagrange interpolating

polynomial from, 504–505 error estimation, 497–502 general form, 495–497 quadratic interpolation, 493–495

Newton’s laws of motion, 118, 827–828 Newton’s method optimization, 352, 365–366,

380, 438–439 Newton’s second law of motion, 11–18, 55, 118,

328, 702 Nodal lines/planes, 891 Nonbasic variables, 397 Nonbinding constraints, 394 Nonideal versus idealized laws, 21 Nonlinear constrained optimization, 352, 401

Excel, 404–408 Mathcad, 413

Nonlinear equations defi ned, 169 fi xed-point iteration, 170–171 linear equations versus, 21, 169 Newton-Raphson method, 171–173 roots of equations, 121, 169–173 systems of equations, 120–121, 231–232,

271–273

cha9792x_ndx_957-976.indd Page 965 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 965 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

966 INDEX

Nonlinear programming optimization, 350 Nonlinear regression, 470, 483–486,

555, 582 Non-self-starting Heun, 228, 229, 707,

722–726, 732, 759–767, 841, 843 Normal distribution, 446 Normalized standard deviate, 651–653 Norms

defi ned, 292 matrix, 292–294 vector, 292–294

nth fi nite divided difference, 495–496 Number systems, 65. See also specifi c number

systems Numerical differentiation, 93–97, 114, 655–670.

See also Differential calculus backward difference approximation,

93–94 centered difference approximation, 95 with computer software, 663–670 control of numerical errors, 105–106 engineering applications, 593–597 error analysis, 102–105 fi nite-divided-difference approximations,

93, 95–97 high-accuracy differentiation formulas,

601, 655–658 polynomial, 179–180 Richardson’s extrapolation, 601, 633,

635–638, 641, 658–660 Numerical integration. See also Integral

calculus advanced methods and additional

references, 695 case studies, 673–683 comparisons, 694–695 with computer software, 663–670 engineering applications, 673–683 important relationships and formulas,

695, 696 Numerical methods of problem solving,

110, 111–112 falling parachutist problem, 17–18 nature of, 15–16

Numerical Recipe library, 48 Numerical stability, 100–101

O Objective function optimization, 348, 350 Octal (base-8) number system, 65 ODEs. See Ordinary differential equations

(ODEs) Ohm’s law, 326

One-dimensional unconstrained optimization, 351, 352, 355–368 Brent’s method, 352, 356, 366–368 golden-section search, 352, 356–363,

427–428, 438 MATLAB, 409–410 multimodal, 355–356 Newton’s method, 352, 365–366, 380,

438–439 parabolic interpolation, 352, 363–365, 438

One-point iteration, 120 One-sided interval, 447 One-step methods, 705, 709–751, 841 Open methods, 120, 145–173, 366

defi ned, 145–146 fi xed-point iteration, 146–151 graphical method, 145

Optimal steepest ascent, 384–386, 585 Optimization, 345–439

additional references, 439 Brent’s method, 352, 356, 366–368, 438 case studies, 416–430 computer methods, 352, 402–413, 419–420 defi ned, 345 engineering applications, 346–350, 352,

416–430 goals/objectives, 352–354 golden-section search, 352, 356–363,

427–428, 438 history, 346 linear programming, 346, 350, 352, 390–401 mathematical background, 350–351 multidimensional unconstrained,

352, 355–356, 370–388 Newton’s method, 352, 365–366, 380,

438–439 noncomputer methods, 346 nonlinear constrained optimization,

352, 401, 404–408, 413 one-dimensional unconstrained, 351, 352,

355–368 parabolic interpolation, 352, 363–365, 438 problem classifi cation, 350–351 scope/preview, 352

Order of polynomials, 119 Ordinary differential equations (ODEs),

176–178, 699–843 advanced methods and additional references,

842–843 boundary-value problems, 705, 707,

782–789, 792–795, 842 case studies, 811–830 components, 699

computer methods, 707, 718–721, 728, 739, 742–744, 749–751, 801–808

defi ned, 699 eigenvalue problems, 707, 789–808 engineering applications, 701–702, 707,

811–830 Euler’s method, 705–707, 710–729, 841,

842, 843 explicit solution technique, 757–759 falling parachutist problem, 701, 709–710 fi nite-difference methods, 707, 786–789 fi rst-order equations, 699 fourth-order Adams, 771–774, 777–779,

841, 842, 843 goals/objectives, 707–708 Heun’s method, 707, 722–726, 728, 729,

732, 759–767, 841, 843 higher-order equations, 699–700, 721, 775–779 implicit solution technique, 707, 755, 757–759 initial-value problems, 705, 781 mathematical background, 703–705 midpoint (improved polygon) method,

707, 726–728, 729, 733–734 Milne’s method, 775–779, 841 multistep methods, 707, 755, 759–779, 841 noncomputer methods for solving, 700–701 one-step methods, 705, 709–751, 841 power methods, 707, 797–800 Ralston’s method, 732–734, 735, 843 Runge-Kutta (RK), 705–707, 729–751,

841, 842, 843 scope/preview, 705–707 second-order equations, 699, 730–734 shooting method, 707, 782, 783–786 stiff systems, 707, 755–759, 804–806, 842 systems of equations, 707, 739–744

Orthogonal, 378 Orthogonal polynomials, 584–585 Overconstrained optimization, 351 Overdamped case, 178 Overdetermined equations, 343 Overfl ow error, 69–70 Overrelaxation, 309

P Parabolic interpolation optimization,

352, 363–365, 438 Parabolic partial differential equations (PDEs),

873–888 alternating-direction implicit (ADI) method,

850, 878–882, 885–888, 931, 932 Crank-Nicolson method, 850, 882–885,

931, 932

cha9792x_ndx_957-976.indd Page 966 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 966 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 967

explicit methods, 874–878, 879, 885, 931 fi nite-difference methods, 846, 847–848,

850, 873–888, 931, 932 heat-conduction equation, 846, 847–848,

873–888 implicit methods, 850, 878–882, 885–888,

931, 932 Laplace equation, 846, 850, 852–866, 922–924 one-dimensional, 884–885, 931 two-dimensional, 885–888, 931

Parameter estimation, 816 Parameters, 11–12, 118, 816

distributed-parameter system, 916 estimation, 816 lumped-parameter systems, 915–916 sinusoidal function, 527–530

Parametric Technology Corporation (PTC), 47 Partial derivatives, 588, 622–623 Partial differential equations (PDEs), 309, 699,

845–932 advanced methods and additional

references, 932 case studies, 915–927 characteristics, 845–846 computer solutions, 850, 869–870, 908–912 defi ned, 845 elliptic equations, 846–847, 850, 852–870,

931, 932 engineering applications, 846–848, 850,

915–927 fi nite-difference methods, 846–848, 850,

852–870, 873–888, 931, 932 fi nite-element methods, 850, 890–908, 931 goals/objectives, 850–851 higher-order temporal approximations,

877–878 hyperbolic equations, 846, 848 important relationships and formulas, 931–932 order of, 845 parabolic equations, 846, 847–848, 850,

873–888, 931, 932 precomputer methods of solving, 848 scope/preview, 849–850

Partial pivoting, 241, 264–269 Pattern directions, 374–375 Pattern searches, 352 Penalty functions, 401 Period, sinusoidal function, 527–528 Phase-plane representation, 819–822 Phase shift, 529 Pivoting, 264–269, 341

complete, 264 division by zero, 258

effect of scaling, 266–269 partial, 241, 264–269 pivot coeffi cient/element, 253–254

Place value, 65 Point-slope method. See Euler’s method Poisson equation, 854, 895–904, 919–924 Polynomial regression, 452, 472–476, 583, 584

algorithm, 475–476 fi t of second-order polynomial, 473–475

Polynomials characteristic equation, 177–178 computing with, 179–182 critically damped case, 178–179 defi ned, 119 defl ation, 180–182 discriminant, 178 eigenvalue problems, 177–178, 795–797 engineering applications, 176–179 evaluation and differentiation, 179–180 factored form, 180 general solution, 177 interpolation, 490–509 Lagrange, 163 Lagrange interpolating, 454, 490, 502–507,

509, 582, 584 Newton-Raphson method, 180, 183, 188 Newton’s divided-difference, 454, 490,

491–502, 504–505, 509, 582, 584 order, 119 ordinary differential equations (ODEs),

176–178, 707 orthogonal, 584–585 overdamped case, 178 polynomial approximation, 83–85 regression, 452, 472–476 roots. See Roots of polynomials synthetic division, 180–181 underdamped case, 179

Populations, estimating properties of, 446–447 Positional notation, 65 Positive defi nite matrix, 304 Postprocessing, fi nite-element methods, 894,

904, 908, 927 Posttest loops, 32–33 Potential energy, 429–430 Powell’s method of optimization, 374–375,

386, 438 Power equations, linear regression of, 470–472 Power methods

defi ned, 797 ordinary differential equations (ODEs), 707,

797–800 Power spectrum, 551–552

Precision, 58–59 Predator-prey models, 818–822 Predictor-corrector approach, 723–724, 762–767 Predictor equation, 722–723 Predictor modifi er, 765–767 Prescriptive models, 346 Pretest loops, 32–33 Product, matrix operations, 236 Programming and software. See Computer

programming and software Propagated truncation error, 713 Propagation problems, 847. See also Hyperbolic

partial differential equations (PDEs); Parabolic partial differential equations (PDEs)

Proportionality, 291 Pseudocode algorithms

adaptive quadrature, 641–642 Bairstow’s method, 190–191 bisection, 133, 134 Brent’s method, 164–166, 366–368 Cholesky decomposition, 304 cubic splines, 520–521 curve fi tting, 454, 568 defi ned, 30 discrete Fourier transform (DFT), 542–544 Euler’s method, 718–721 Excel VBA versus, 41 fast Fourier transform (FFT), 549–550 fi xed-point iteration, 150–151 forward elimination, 254 function that involves differential equation, 38 Gauss-Seidel (Liebmann) method, 309–310 for generic iterative calculation, 62–63 golden-section-search optimization, 361,

362, 427–428 linear regression, 464–465, 478–479 logical representation, 30–37 LU decomposition, 282, 284–285, 286–287 MATLAB versus, 45 matrix inverse, 289–290 modifi ed false-position method, 140 Müller’s method, 186–187 multiple linear regression, 478–479 Newton’s divided-difference interpolating

polynomials, 499–502 optimization, 427–428 partial pivoting, 266, 268 polynomial regression, 475–476 Romberg integration, 639–640 roots of quadratic equation, 34–37, 77 Runge-Kutta (RK) method, 749–750 Simpson’s rules, 622, 623, 626–627 Thomas algorithm, 301–302

cha9792x_ndx_957-976.indd Page 967 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 967 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

968 INDEX

Q QR factorization, 482 QR method (Francis), 801 Quadratic equation, algorithm for roots, 34–37 Quadratic interpolation, 493–495 Quadratic programming, 350 Quadratic splines, 514–517 Quadrature methods, 592 Quantizing errors, 70–71, 72, 75 Quasi-Newton methods of optimization,

352, 388, 401, 439 Quotient difference (QD) algorithm, 228

R Ralston’s method, 732–734, 735, 843 Random search method of optimization,

352, 371–373 Rate equation, 699 Reaction kinetics, 816 Regression. See Linear regression; Polynomial

regression Relative error, 100 Relaxation, 309, 911–912 Remainder, 61, 62, 114

Taylor series, 87–89, 114 Repetition, in logical representation, 31–34 Residual error, 457, 461–464 Response, 35 Richardson’s extrapolation, 601, 633, 635–638,

641, 658–660 Ridder method, root of polynomials, 199 Romberg integration, 601, 633, 634–640, 641,

679, 694, 696 Root polishing, 182 Roots of equations, 117–229

advanced methods and additional references, 227–229

Bairstow’s method, 187–191, 226 bisection method, 120, 127–135, 226, 228,

356–357 bracketing methods. See Bracketing methods Brent’s method, 120–121, 162–166, 226, 227 case studies, 122, 204–215 computer methods, 126–127, 192–201 engineering applications, 118–119, 122,

176–179, 204–215 false-position method, 120, 135–141, 226, 228 fi xed-point iteration, 146–151, 226 goals/objectives, 122 graphical methods, 117, 120–121, 123–127,

145, 146–151, 157–166, 226

important relationships and formulas, 227, 228

incremental searches/determined incremental guesses, 127, 141

mathematical background, 119–120, 179–182

Müller’s method, 183–187, 199, 226 multiple roots, 125, 166–169 Newton-Raphson method, 120–121,

151–157, 166–167, 226, 227, 228, 365 noncomputer methods, 117 nonlinear equations, 121, 169–173 open methods, 145–173 optimization and, 345 other methods, 192 polynomials, 120, 121–122, 176–192 scope/preview, 120–122 secant method, 120, 157–162, 166–167,

226, 228 as zeros of equation, 117

Roots of polynomials, 120, 121–122, 176–192 Bairstow’s method, 187–191 Brent’s method, 199 computer methods, 192–201 conventional methods, 182–183 Jenkins-Traub method, 192, 229 Laguerre’s method, 192, 199, 229 Müller’s method, 183–187, 199, 226 polynomial defl ation, 180–182

Rounding, 71 Round-off errors, 65–79

adding a large and a small number, 75–76 arithmetic manipulation of computer

numbers, 73–79 common arithmetic operations, 73–74 computer representation of numbers, 65–73 defi ned, 56, 59 Euler’s method, 713 extended precision, 72–73 fl oating-point representation, 67–72 Gauss elimination, 259 integer representation, 65–67 iterative refi nement, 296–297 large computations, 74–75 linear algebraic equations, 259 number systems, 65 numerical differentiation, 103–105 signifi cant digits and, 57 smearing, 77–79 subtractive cancellation, 76–77 total numerical error, 101–106

Row-sum norms, 294 Row vectors, 234 Runge-Kutta Fehlberg method, 747–748, 749 Runge-Kutta (RK) methods, 705–707, 729–751,

841, 842 adaptive, 707, 744–751 adaptive step-size control, 745,

748–749, 768 Cash-Karp RK method, 747, 749–750 comparison, 737–739 computer algorithms, 739 embedded, 747 fourth-order, 735–737, 741–742, 743, 746,

750–751, 843 higher-order, 737–739 Runge-Kutta Fehlberg method,

747–748, 749 second-order, 730–734 systems of equations, 740–742 third-order, 734–735

S Saddle, 379 Samples, estimating properties of, 446–447 Sande-Tukey algorithm, 545, 546–550 Scaling

effect of scale on determinant in ill-conditioned systems, 261–263

effect on pivoting and round-off, 266–269 Secant method, 120, 157–162

algorithm, 161 false-position method versus, 158–160 graphical method, 157, 159, 160, 162, 228 modifi ed, 161–162, 226 multiple roots, 166–167 root of polynomials, 199

Second Adams-Moulton formula, 773–774 Second derivative, 588, 656–657 Second fi nite divided difference, 495 Second forward fi nite divided difference, 96–97 Second-order approximation, 82–83, 86 Second-order closed Adams formula, 773–774 Second-order equations, 699, 730–734 Selection, in logical representation, 30–31 Sensitivity analysis, 21, 42–43 Sentinel variables, 309–310 Sequence, in logical representation, 30 Shadow price, 424 Shooting method, 707, 782, 783–786 Signed magnitude method, 65–66 Signifi cance level, 448 Signifi cand, 67–68

cha9792x_ndx_957-976.indd Page 968 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 968 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

INDEX 969

Signifi cant fi gures/digits, 56–57 Simple statistics, 443–446 Simplex method, 346, 352, 396–401

algebraic solution, 396–397 implementation, 398–401 slack variables, 396

Simpson’s 1/3 rule, 600, 615–620, 676, 694, 696 computer algorithms, 623 derivation and error estimate, 616 multiple-application, 618–620, 641 single-application, 617–618 with unequal segments, 625–627

Simpson’s 3/8 rule, 600, 620–622, 694, 696 computer algorithms, 623 with unequal segments, 625–627

Simultaneous overrelaxation, 309 Single-value decomposition, 585 Single-variable optimization, 356 Singular systems, 247, 263–264 Sinusoidal functions, 527–533

least-squares fi t of sinusoid, 530–533 parameters, 527–530

Slack variables, 396 Smearing, 77–79 Software. See Computer programming and

software Spline functions, 511, 585 Spline interpolation, 454, 511–521

cubic splines, 511, 517–521, 559–560, 582, 584, 695

engineering applications, 567–568 linear splines, 511–514 quadratic splines, 514–517 splines, defi ned, 511

Spread around the mean, 462 Spread around the regression line, 462 Spreadsheets. See Excel Square matrices, 235 Stability

defi ned, 877 error propagation, 100–101 of numerical methods of problem solving,

111–112 Standard deviation, 444 Standard error of the estimate, 462 Standard normal estimate, 448–449 Standard normal random variable, 449 Statistical inference, 447, 449 Statistics, 443–452

estimation of confi dence interval, 446–452, 481–482

least-squares theory, 480–483

maximum likelihood principle, 461–462 normal distribution, 446 simple statistics, 443–446

Steady-state computation, 19, 319–322. See also Elliptic partial differential equations (PDEs)

Steepest ascent/descent optimization, 352, 381–386 optimal steepest ascent, 384–386, 585 using gradient to evaluate, 377–378

Stiffness matrix, fi nite-element methods, 894, 926 Stiff ordinary differential equations (ODEs),

707, 755–759, 804–806, 842 Euler’s method, 756, 757 stiff system, defi ned, 755

Stimulus-response computations, 290–291 Stopping criterion, 62, 63, 114 Strip method, 592–593, 604 Structured programming, 28–37

defi ned, 29 fl owcharts, 29–30 logical representation, 30–37 pseudocode, 30–37

Subroutines, modular programming, 37 Subtraction, 74

matrix operations, 236 subtractive cancellation, 76–77

Successive overrelaxation, 309 Summation, 588 Superposition, 291 Swamee-Jain equation, 212 Symmetric matrices, 300 Synthetic division, 180–181 Systems of equations

nonlinear equations, 120–121, 231–232, 271–273

ordinary differential equations (ODEs), 707, 739–744

T Tableau, 398–401 Taylor series, 81–97, 114. See also

Finite-difference methods approximation of polynomial, 83–85 backward difference approximation, 93–94 centered fi nite divided-difference

approximation, 94, 95 defi ned, 81 derivative mean-value theorem, 88 error propagation, 97–101 to estimate error for Euler’s method,

714–715 to estimate truncation errors, 89–93, 721

expansion of Newton-Raphson method, 272 expansion of Newton’s divided-difference

interpolating polynomials, 497–498 expansions, 85–89, 114, 272, 497–498 fi nite difference approximations, 96–97 fi nite-divided-difference approximations,

93, 95–96, 655–658 fi rst forward difference, 93 fi rst-order approximation, 81–82, 84, 86,

92–93 fi rst theorem of mean for integrals, 82 infi nite number of derivatives, 86–87 linear trend, 82–83 nonlinearity, 89–93, 483 numerical differentiation, 93–97 remainder, 87–89, 114 second-order approximation, 82–83, 86 second theorem of mean for integrals, 82 step size, 89–93 Taylor’s theorem/formula, 81, 82 zero-order approximation, 81, 84, 86

t distribution, 450–451 Terminal velocity, 15 Thermal conductivity, 853 Thermal diffusivity, 853 Thermocline, 567 Third derivative, 656–657 Third-order methods, 734–735 Thomas algorithm, 301–302 Time domains, 536–540 Time plane, 536–537 Time-variable (transient) computation, 18 Topography, 351 Total numerical error, 101–106 Total sum of the squares, 462–463 Total value, 588 Trace, matrix operations, 239 Transcendental function, 120 Transient computation, 18 Transpose, matrix operations, 239 Trapezoidal rule, 600, 605–615, 623, 624–627,

676, 694, 696, 726 computer algorithms, 612–615 as corrector, 759–760 error/error correction, 607–609, 636–637 multiple-application, 609–612, 696 single-application, 608–609 with unequal segments, 624–625

Trend analysis, 442–443 Tridiagonal systems, 241–243, 301–302 Triple roots, 166, 167 True derivative approximation, 94

cha9792x_ndx_957-976.indd Page 969 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 969 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

970 INDEX

True error, 59, 98, 104, 114 True mean, 449 True percent relative error, 59, 61, 64, 114 Truncation errors. See also Discretization,

fi nite-element methods defi ned, 56, 59, 81 Euler’s method, 712, 721 numerical differentiation, 103–105 per-step, 765 Taylor series to estimate, 89–93, 721.

See also Taylor series total numerical error, 101–106 types, 712–713

Twiddle factors, 547 2’s complement, 67 Two-dimensional interpolation, 522–523 Two-sided interval, 447–448

U Uncertainty, 58–59, 107, 662 Unconditionally stable, 757 Unconstrained optimization, 351, 352

multidimensional. See Multidimensional unconstrained optimization

one-dimensional. See One-dimensional unconstrained optimization

Underdamped case, 179 Underdetermined equations, 342–343, 396 Underrelaxation, 309 Underspecifi ed equations, 396 Uniform matrix norms, 294 Uniform vector norms, 294 Unimodal optimization, 356–357 Univariate search method, 352, 373

V Van der Waals equation, 205 Variable metric methods of optimization,

352, 388 Variables

basic, 397 dependent, 11–12, 118, 699 design, 35, 350 entering, 398–399 independent, 11–12, 118, 699 leaving, 398–399 lumped-variable systems, 232, 233, 311 single-variable optimization, 356 slack, 396 standard normal random, 449

Variable step size, 768 Variance, 444, 449

Variation, coeffi cient of, 445 Vector norms, 292–294 Very large numbers, fl oating-point

representation, 68 Visual Basic Editor (VBE), 40–43 Voltage balance, 20 Volume-integral approach, 866–869 Volume integrals, 596

W Wave equation, 846, 848 Well-conditioned systems, 259 WHILE structure, 33

Z Zero, division by, 258 Zero-order approximation, 81, 84, 86

cha9792x_ndx_957-976.indd Page 970 10/12/13 3:31 PM user-f467 cha9792x_ndx_957-976.indd Page 970 10/12/13 3:31 PM user-f467 /207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles/207/MH02101/cha9792x_disk1of1/007339792x/cha9792x_pagefiles

  • COVER
  • TITLE
  • COPYRIGHT
  • CONTENTS
  • PREFACE
  • ABOUT THE AUTHORS
  • PART ONE
    • MODELING, COMPUTERS, AND ERROR ANALYSIS
      • PT1.1 Motivation
      • PT1.2 Mathematical Background
      • PT1.3 Orientation
    • CHAPTER 1 Mathematical Modeling and Engineering Problem Solving
      • 1.1 A Simple Mathematical Model
      • 1.2 Conservation Laws and Engineering
      • Problems
    • CHAPTER 2 Programming and Software
      • 2.1 Packages and Programming
      • 2.2 Structured Programming
      • 2.3 Modular Programming
      • 2.4 Excel
      • 2.5 MATLAB
      • 2.6 Mathcad
      • 2.7 Other Languages and Libraries
      • Problems
    • CHAPTER 3 Approximations and Round-Off Errors
      • 3.1 Significant Figures
      • 3.2 Accuracy and Precision
      • 3.3 Error Definitions
      • 3.4 Round-Off Errors
      • Problems
    • CHAPTER 4 Truncation Errors and the Taylor Series
      • 4.1 The Taylor Series
      • 4.2 Error Propagation
      • 4.3 Total Numerical Error
      • 4.4 Blunders, Formulation Errors, and Data Uncertainty
      • Problems
    • EPILOGUE: PART ONE
      • PT1.4 Trade-Offs
      • PT1.5 Important Relationships and Formulas
      • PT1.6 Advanced Methods and Additional References
  • PART TWO
    • CHAPTER 5 Bracketing Methods
      • 5.1 Graphical Methods
      • 5.2 The Bisection Method
      • 5.3 The False-Position Method
      • 5.4 Incremental Searches and Determining Initial Guesses
      • Problems
    • CHAPTER 6 Open Methods
      • 6.1 Simple Fixed-Point Iteration
      • 6.2 The Newton-Raphson Method
      • 6.3 The Secant Method
      • 6.4 Brent's Method
      • 6.5 Multiple Roots
      • 6.6 Systems of Nonlinear Equations
      • Problems
    • CHAPTER 7 Roots of Polynomials
      • 7.1 Polynomials in Engineering and Science
      • 7.2 Computing with Polynomials
      • 7.3 Conventional Methods
      • 7.4 Müller's Method
      • 7.5 Bairstow's Method
      • 7.6 Other Methods
      • 7.7 Root Location with Software Packages
      • Problems
    • CHAPTER 8 Case Studies: Roots of Equations
      • 8.1 Ideal and Nonideal Gas Laws (Chemical/Bio Engineering)
      • 8.2 Greenhouse Gases and Rainwater (Civil/Environmental Engineering)
      • 8.3 Design of an Electric Circuit (Electrical Engineering)
      • 8.4 Pipe Friction (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART TWO
      • PT2.4 Trade-Offs
      • PT2.5 Important Relationships and Formulas
      • PT2.6 Advanced Methods and Additional References
    • ROOTS OF EQUATIONS
      • PT2.1 Motivation
      • PT2.2 Mathematical Background
      • PT2.3 Orientation
  • PART THREE
    • LINEAR ALGEBRAIC EQUATIONS
      • PT3.1 Motivation
      • PT3.2 Mathematical Background
      • PT3.3 Orientation
    • CHAPTER 9 Gauss Elimination
      • 9.1 Solving Small Numbers of Equations
      • 9.2 Naive Gauss Elimination
      • 9.3 Pitfalls of Elimination Methods
      • 9.4 Techniques for Improving Solutions
      • 9.5 Complex Systems
      • 9.6 Nonlinear Systems of Equations
      • 9.7 Gauss-Jordan
      • 9.8 Summary
      • Problems
    • CHAPTER 10 LU Decomposition and Matrix Inversion
      • 10.1 LU Decomposition
      • 10.2 The Matrix Inverse
      • 10.3 Error Analysis and System Condition
      • Problems
    • CHAPTER 11 Special Matrices and Gauss-Seidel
      • 11.1 Special Matrices
      • 11.2 Gauss-Seidel
      • 11.3 Linear Algebraic Equations with Software Packages
      • Problems
    • CHAPTER 12 Case Studies: Linear Algebraic Equations
      • 12.1 Steady-State Analysis of a System of Reactors (Chemical/Bio Engineering)
      • 12.2 Analysis of a Statically Determinate Truss (Civil/Environmental Engineering)
      • 12.3 Currents and Voltages in Resistor Circuits (Electrical Engineering)
      • 12.4 Spring-Mass Systems (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART THREE
      • PT3.4 Trade-Offs
      • PT3.5 Important Relationships and Formulas
      • PT3.6 Advanced Methods and Additional References
  • PART FOUR
    • OPTIMIZATION
      • PT4.1 Motivation
      • PT4.2 Mathematical Background
      • PT4.3 Orientation
    • CHAPTER 13 One-Dimensional Unconstrained Optimization
      • 13.1 Golden-Section Search
      • 13.2 Parabolic Interpolation
      • 13.3 Newton's Method
      • 13.4 Brent's Method
      • Problems
    • CHAPTER 14 Multidimensional Unconstrained Optimization
      • 14.1 Direct Methods
      • 14.2 Gradient Methods
      • Problems
    • CHAPTER 15 Constrained Optimization
      • 15.1 Linear Programming
      • 15.2 Nonlinear Constrained Optimization
      • 15.3 Optimization with Software Packages
      • Problems
    • CHAPTER 16 Case Studies: Optimization
      • 16.1 Least-Cost Design of a Tank (Chemical/Bio Engineering)
      • 16.2 Least-Cost Treatment of Wastewater (Civil/Environmental Engineering)
      • 16.3 Maximum Power Transfer for a Circuit (Electrical Engineering)
      • 16.4 Equilibrium and Minimum Potential Energy (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART FOUR
      • PT4.4 Trade-Offs
      • PT4.5 Additional References
  • PART FIVE
    • CURVE FITTING
      • PT5.1 Motivation
      • PT5.2 Mathematical Background
      • PT5.3 Orientation
    • CHAPTER 17 Least-Squares Regression
      • 17.1 Linear Regression
      • 17.2 Polynomial Regression
      • 17.3 Multiple Linear Regression
      • 17.4 General Linear Least Squares
      • 17.5 Nonlinear Regression
      • Problems
    • CHAPTER 18 Interpolation
      • 18.1 Newton's Divided-Difference Interpolating Polynomials
      • 18.2 Lagrange Interpolating Polynomials
      • 18.3 Coefficients of an Interpolating Polynomial
      • 18.4 Inverse Interpolation
      • 18.5 Additional Comments
      • 18.6 Spline Interpolation
      • 18.7 Multidimensional Interpolation
      • Problems
    • CHAPTER 19 Fourier Approximation
      • 19.1 Curve Fitting with Sinusoidal Functions
      • 19.2 Continuous Fourier Series
      • 19.3 Frequency and Time Domains
      • 19.4 Fourier Integral and Transform
      • 19.5 Discrete Fourier Transform (DFT)
      • 19.6 Fast Fourier Transform (FFT)
      • 19.7 The Power Spectrum
      • 19.8 Curve Fitting with Software Packages
      • Problems
    • CHAPTER 20 Case Studies: Curve Fitting
      • 20.1 Linear Regression and Population Models (Chemical/Bio Engineering)
      • 20.2 Use of Splines to Estimate Heat Transfer (Civil/Environmental Engineering)
      • 20.3 Fourier Analysis (Electrical Engineering)
      • 20.4 Analysis of Experimental Data (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART FIVE
      • PT5.4 Trade-Offs
      • PT5.5 Important Relationships and Formulas
      • PT5.6 Advanced Methods and Additional References
  • PART SIX
    • NUMERICAL DIFFERENTIATION AND INTEGRATION
      • PT6.1 Motivation
      • PT6.2 Mathematical Background
      • PT6.3 Orientation
    • CHAPTER 21 Newton-Cotes Integration Formulas
      • 21.1 The Trapezoidal Rule
      • 21.2 Simpson's Rules
      • 21.3 Integration with Unequal Segments
      • 21.4 Open Integration Formulas
      • 21.5 Multiple Integrals
      • Problems
    • CHAPTER 22 Integration of Equations
      • 22.1 Newton-Cotes Algorithms for Equations
      • 22.2 Romberg Integration
      • 22.3 Adaptive Quadrature
      • 22.4 Gauss Quadrature
      • 22.5 Improper Integrals
      • Problems
    • CHAPTER 23 Numerical Differentiation
      • 23.1 High-Accuracy Differentiation Formulas
      • 23.2 Richardson Extrapolation
      • 23.3 Derivatives of Unequally Spaced Data
      • 23.4 Derivatives and Integrals for Data with Errors
      • 23.5 Partial Derivatives
      • 23.6 Numerical Integration/Differentiation with Software Packages
      • Problems
    • CHAPTER 24 Case Studies: Numerical Integration and Differentiation
      • 24.1 Integration to Determine the Total Quantity of Heat (Chemical/Bio Engineering)
      • 24.2 Effective Force on the Mast of a Racing Sailboat (Civil/Environmental Engineering)
      • 24.3 Root-Mean-Square Current by Numerical Integration (Electrical Engineering)
      • 24.4 Numerical Integration to Compute Work (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART SIX
      • PT6.4 Trade-Offs
      • PT6.5 Important Relationships and Formulas
      • PT6.6 Advanced Methods and Additional References
  • PART SEVEN
    • ORDINARY DIFFERENTIAL EQUATIONS
      • PT7.1 Motivation
      • PT7.2 Mathematical Background
      • PT7.3 Orientation
    • CHAPTER 25 Runge-Kutta Methods
      • 25.1 Euler's Method
      • 25.2 Improvements of Euler's Method
      • 25.3 Runge-Kutta Methods
      • 25.4 Systems of Equations
      • 25.5 Adaptive Runge-Kutta Methods
      • Problems
    • CHAPTER 26 Stiffness and Multistep Methods
      • 26.1 Stiffness
      • 26.2 Multistep Methods
      • Problems
    • CHAPTER 27 Boundary-Value and Eigenvalue Problems
      • 27.1 General Methods for Boundary-Value Problems
      • 27.2 Eigenvalue Problems
      • 27.3 Odes and Eigenvalues with Software Packages
      • Problems
    • CHAPTER 28 Case Studies: Ordinary Differential Equations
      • 28.1 Using ODEs to Analyze the Transient Response of a Reactor (Chemical/Bio Engineering)
      • 28.2 Predator-Prey Models and Chaos (Civil/Environmental Engineering)
      • 28.3 Simulating Transient Current for an Electric Circuit (Electrical Engineering)
      • 28.4 The Swinging Pendulum (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART SEVEN
      • PT7.4 Trade-Offs
      • PT7.5 Important Relationships and Formulas
      • PT7.6 Advanced Methods and Additional References
  • PART EIGHT
    • PARTIAL DIFFERENTIAL EQUATIONS
      • PT8.1 Motivation
      • PT8.2 Orientation
    • CHAPTER 29 Finite Difference: Elliptic Equations
      • 29.1 The Laplace Equation
      • 29.2 Solution Technique
      • 29.3 Boundary Conditions
      • 29.4 The Control-Volume Approach
      • 29.5 Software to Solve Elliptic Equations
      • Problems
    • CHAPTER 30 Finite Difference: Parabolic Equations
      • 30.1 The Heat-Conduction Equation
      • 30.2 Explicit Methods
      • 30.3 A Simple Implicit Method
      • 30.4 The Crank-Nicolson Method
      • 30.5 Parabolic Equations in Two Spatial Dimensions
      • Problems
    • CHAPTER 31 Finite-Element Method
      • 31.1 The General Approach
      • 31.2 Finite-Element Application in One Dimension
      • 31.3 Two-Dimensional Problems
      • 31.4 Solving PDEs with Software Packages
      • Problems
    • CHAPTER 32 Case Studies: Partial Differential Equations
      • 32.1 One-Dimensional Mass Balance of a Reactor (Chemical/Bio Engineering)
      • 32.2 Deflections of a Plate (Civil/Environmental Engineering)
      • 32.3 Two-Dimensional Electrostatic Field Problems (Electrical Engineering)
      • 32.4 Finite-Element Solution of a Series of Springs (Mechanical/Aerospace Engineering)
      • Problems
    • EPILOGUE: PART EIGHT
      • PT8.3 Trade-Offs
      • PT8.4 Important Relationships and Formulas
      • PT8.5 Advanced Methods and Additional References
  • APPENDIX A: THE FOURIER SERIES
  • APPENDIX B: GETTING STARTED WITH MATLAB
  • APPENDIX C: GETTING STARTED WITH MATHCAD
  • BIBLIOGRAPHY
  • INDEX
    • A
    • B
    • C
    • D
    • E
    • F
    • G
    • H
    • I
    • J
    • K
    • L
    • M
    • N
    • O
    • P
    • Q
    • R
    • S
    • T
    • U
    • V
    • W
    • Z