Preface |
|
vii | |
Acknowledgments |
|
xi | |
|
|
|
|
3 | (6) |
|
Taking Rocket Science Beyond the Frontiers of Space |
|
|
3 | (2) |
|
Why Drugs, Corruption, and Terror? |
|
|
5 | (2) |
|
Questions Optimal Control Can Answer |
|
|
7 | (2) |
|
Continuous-Time Dynamical Systems |
|
|
9 | (92) |
|
Nonlinear Dynamical Modeling |
|
|
9 | (1) |
|
|
10 | (4) |
|
A One-Dimensional Corruption Model |
|
|
14 | (3) |
|
Dynamical Systems as ODEs |
|
|
17 | (10) |
|
|
19 | (2) |
|
Invariant Sets and Stability |
|
|
21 | (4) |
|
|
25 | (1) |
|
Linearization and the Variational Equation |
|
|
26 | (1) |
|
Stability Analysis of a One-Dimensional Terror Model |
|
|
27 | (3) |
|
ODEs in Higher Dimensions |
|
|
30 | (21) |
|
|
31 | (11) |
|
Autonomous Nonlinear ODEs |
|
|
42 | (9) |
|
Stability Behavior in a Descriptive Model of Drug Demand |
|
|
51 | (4) |
|
Introduction to Bifurcation Theory |
|
|
55 | (13) |
|
Terminology and Key Ideas of Bifurcation Theory |
|
|
56 | (1) |
|
Normal Forms and the Center Manifold: The Tools of Bifurcation Theory |
|
|
57 | (6) |
|
Local Bifurcations in One Dimension |
|
|
63 | (5) |
|
Bifurcation Analysis of a One-Dimensional Drug Model |
|
|
68 | (3) |
|
The Poincare-Andronov-Hopf Bifurcation |
|
|
71 | (3) |
|
Higher-Dimensional Bifurcation Analysis of a Drug Model |
|
|
74 | (4) |
|
|
78 | (23) |
|
Stability of Limit Cycles |
|
|
78 | (7) |
|
|
85 | (4) |
|
|
89 | (7) |
|
Notes and Further Reading |
|
|
96 | (5) |
|
Part II Applied Optimal Control |
|
|
|
Tour d'Horizon: Optimal Control |
|
|
101 | (88) |
|
|
101 | (3) |
|
A Standard Optimal Control Problem |
|
|
104 | (4) |
|
The Maximum Principle of Optimal Control Theory |
|
|
108 | (19) |
|
Pontryagin's Maximum Principle |
|
|
108 | (5) |
|
|
113 | (2) |
|
The Maximum Principle for Variable Terminal Time |
|
|
115 | (2) |
|
Economic Interpretation of the Maximum Principle |
|
|
117 | (2) |
|
|
119 | (3) |
|
Existence of an Optimal Solution |
|
|
122 | (2) |
|
How to Solve an Optimal Control Problem: A Simple Consumption vs. Investment Model |
|
|
124 | (3) |
|
The Principle of Optimality |
|
|
127 | (4) |
|
The Hamilton-Jacobi-Bellman Equation |
|
|
127 | (3) |
|
A Proof of the Maximum Principle |
|
|
130 | (1) |
|
|
131 | (11) |
|
The Most Rapid Approach Path (MRAP) |
|
|
134 | (3) |
|
An Example From Drug Control that Excludes Singular Arcs |
|
|
137 | (2) |
|
An Example From Terror Control with an MRAP Solution |
|
|
139 | (3) |
|
The Maximum Principle With Inequality Constraints |
|
|
142 | (13) |
|
|
144 | (3) |
|
|
147 | (7) |
|
|
154 | (1) |
|
|
155 | (4) |
|
Definitions of Optimality for Infinite Horizon Problems |
|
|
155 | (1) |
|
Maximum Principle for Infinite Time Horizon Problems |
|
|
156 | (3) |
|
|
159 | (1) |
|
Discounted Autonomous Infinite Horizon Models |
|
|
159 | (9) |
|
|
160 | (5) |
|
The Ramsey Model for an Infinite Time Horizon |
|
|
165 | (2) |
|
Structural Results on One-State Discounted, Autonomous Systems |
|
|
167 | (1) |
|
An Optimal Control Model of a Drug Epidemic |
|
|
168 | (21) |
|
|
168 | (2) |
|
|
170 | (6) |
|
|
176 | (1) |
|
|
177 | (6) |
|
Notes and Further Reading |
|
|
183 | (6) |
|
The Path to Deeper Insight: From Lagrange to Pontryagin |
|
|
189 | (48) |
|
Introductory Remarks on Optimization |
|
|
189 | (8) |
|
|
190 | (1) |
|
|
190 | (2) |
|
A Simple Maximization Problem |
|
|
192 | (3) |
|
Finite-Dimensional Approximation of an Infinite-Dimensional Problem |
|
|
195 | (2) |
|
|
197 | (17) |
|
Basic Theorems and Definitions |
|
|
198 | (4) |
|
Theory and Geometric Interpretation of Lagrange and Karush-Kuhn-Tucker |
|
|
202 | (6) |
|
The Envelope Theorem and the Lagrange Multiplier |
|
|
208 | (2) |
|
The Discrete-Time Maximum Principle as a Static Maximization Problem |
|
|
210 | (4) |
|
The Calculus of Variations |
|
|
214 | (9) |
|
A Simple Variational Example |
|
|
214 | (2) |
|
|
216 | (2) |
|
Deriving the Euler Equation and Weierstrass-Erdmann Conditions |
|
|
218 | (5) |
|
Proving the Continuous-Time Maximum Principle |
|
|
223 | (14) |
|
The Continuous-Time Maximum Principle Revisited |
|
|
223 | (4) |
|
Necessary Conditions at Junction Points |
|
|
227 | (4) |
|
|
231 | (3) |
|
Notes and Further Reading |
|
|
234 | (3) |
|
Multiple Equilibria, Points of Indifference, and Thresholds |
|
|
237 | (42) |
|
Occurrence of Multiple Equilibria |
|
|
238 | (1) |
|
|
239 | (5) |
|
Finite vs. Infinite Time Horizon Models |
|
|
239 | (4) |
|
Discounted Autonomous Models for an Infinite Time Horizon |
|
|
243 | (1) |
|
|
244 | (8) |
|
Existence and Stability of the Equilibria |
|
|
245 | (2) |
|
Determining the Optimal Vector Field and the Optimal Costate Rule |
|
|
247 | (5) |
|
Defining Indifference and DNSS Points |
|
|
252 | (8) |
|
Multiplicity and Separability |
|
|
253 | (1) |
|
|
254 | (2) |
|
Conclusions from the Definitions |
|
|
256 | (4) |
|
Revisiting the Typical Example |
|
|
260 | (6) |
|
Eradication vs. Accommodation in an Optimal Control Model of a Drug Epidemic |
|
|
266 | (13) |
|
|
269 | (3) |
|
Notes and Further Reading |
|
|
272 | (7) |
|
|
|
Higher-Dimensional Models |
|
|
279 | (48) |
|
Controlling Drug Consumption |
|
|
280 | (16) |
|
Model of Controlled Drug Demand |
|
|
280 | (3) |
|
Deriving the Canonical System |
|
|
283 | (3) |
|
The Endemic Level of Drug Demand |
|
|
286 | (1) |
|
Optimal Dynamic Policy away from the Endemic State |
|
|
287 | (5) |
|
Optimal Policies for Different Phases of a Drug Epidemic |
|
|
292 | (4) |
|
Corruption in Governments Subject to Popularity Constraints |
|
|
296 | (12) |
|
The Modeled Incentive for Being Corrupt |
|
|
297 | (2) |
|
|
299 | (1) |
|
Insights About the Incentive to Be Corrupt |
|
|
300 | (2) |
|
Is Periodic Behavior Caused by Rational Optimization? |
|
|
302 | (6) |
|
Is It Important to Manage Public Opinion While Fighting Terrorism? |
|
|
308 | (19) |
|
What One Should Know when Fighting Terrorism |
|
|
309 | (1) |
|
Derivation of the Canonical System |
|
|
310 | (1) |
|
|
311 | (3) |
|
Optimal Strategy for a Small Terror Organization |
|
|
314 | (2) |
|
|
316 | (7) |
|
Notes and Further Reading |
|
|
323 | (4) |
|
Numerical Methods for Discounted Systems of Infinite Horizon |
|
|
327 | (58) |
|
|
327 | (5) |
|
Problem Formulation and Assumptions |
|
|
328 | (1) |
|
|
329 | (1) |
|
Numerical Methods for Solving Optimal Control Problems |
|
|
330 | (1) |
|
Boundary Value Problems from Optimal Control |
|
|
330 | (2) |
|
|
332 | (10) |
|
|
333 | (5) |
|
Continuing the Solution of a BVP |
|
|
338 | (4) |
|
The Canonical System Without Active Constraints |
|
|
342 | (1) |
|
Calculating Long-Run Optimal Solutions |
|
|
343 | (6) |
|
|
344 | (2) |
|
|
346 | (3) |
|
Continuing the Optimal Solution: Calculating the Stable Manifold |
|
|
349 | (10) |
|
Stable Manifold of an Equilibrium |
|
|
350 | (4) |
|
Stable Manifold of Limit Cycles |
|
|
354 | (5) |
|
Optimal Control Problems with Active Constraints |
|
|
359 | (7) |
|
The Form of the Canonical System for Mixed Path Constraints |
|
|
360 | (1) |
|
The Form of the Canonical System for Pure State Constraints |
|
|
360 | (2) |
|
Solutions Exhibiting Junction Points |
|
|
362 | (4) |
|
|
366 | (2) |
|
|
366 | (2) |
|
|
368 | (1) |
|
Retrieving Heteroclinic Connections |
|
|
368 | (2) |
|
Locating a Heteroclinic Connection |
|
|
368 | (1) |
|
Continuing a Heteroclinic Connection in Parameter Space |
|
|
369 | (1) |
|
Numerical Example from Drug Control |
|
|
370 | (15) |
|
Stating the Necessary Conditions |
|
|
370 | (2) |
|
Equilibria of the Canonical System |
|
|
372 | (1) |
|
|
372 | (1) |
|
Optimal Vector Field for ν = 4,000 |
|
|
373 | (4) |
|
Optimal Vector Field for ν = 12,000 |
|
|
377 | (3) |
|
|
380 | (2) |
|
Notes and Further Reading |
|
|
382 | (3) |
|
Extensions of the Maximum Principle |
|
|
385 | (120) |
|
Multi-Stage Optimal Control Problems |
|
|
386 | (5) |
|
Necessary Optimality Conditions for Two-Stage Control Problems |
|
|
386 | (1) |
|
Two-Stage Models of Drug Control |
|
|
387 | (1) |
|
Counter-Terror Measures in a Multi-Stage Scenario |
|
|
388 | (3) |
|
|
391 | (26) |
|
|
392 | (2) |
|
|
394 | (3) |
|
Tractable Game Structures |
|
|
397 | (1) |
|
A Corrupt Politician vs. the Tabloid Press |
|
|
397 | (7) |
|
|
404 | (3) |
|
A Post September 11th Game on Terrorism |
|
|
407 | (10) |
|
|
417 | (5) |
|
A Maximum Principle for Distributed Parameter Systems |
|
|
419 | (1) |
|
Age-Structured Drug Initiation |
|
|
420 | (2) |
|
Further Optimal Control Issues |
|
|
422 | (21) |
|
|
422 | (2) |
|
Stochastic Optimal Control |
|
|
424 | (1) |
|
Impulse Control and Jumps in the State Variables |
|
|
425 | (1) |
|
|
426 | (1) |
|
|
426 | (10) |
|
Notes and Further Reading |
|
|
436 | (7) |
|
|
|
A Mathematical Background |
|
|
443 | (40) |
|
A.1 General Notation and Functions |
|
|
443 | (4) |
|
A.2 Finite-Dimensional Vector Spaces |
|
|
447 | (1) |
|
A.2.1 Vector Spaces, Linear Dependence, and Basis |
|
|
447 | (3) |
|
A.2.2 Linear Transformations and Matrices |
|
|
450 | (3) |
|
A.2.3 Inverse Matrices and Linear Equations |
|
|
453 | (2) |
|
|
455 | (2) |
|
A.2.5 Linear Form and Dual Space |
|
|
457 | (2) |
|
A.2.6 Eigenvalues and Eigenvectors |
|
|
459 | (2) |
|
A.2.7 Euclidean Vector Space Rn |
|
|
461 | (2) |
|
A.3 Topology and Calculus |
|
|
463 | (1) |
|
A.3.1 Open Set, Neighborhood, and Convergence |
|
|
463 | (1) |
|
A.3.2 Continuity and Differentiability |
|
|
464 | (7) |
|
A.3.3 Maximization of Real-Valued Functions in Rn |
|
|
471 | (2) |
|
|
473 | (2) |
|
A.3.5 Taylor Theorem and Implicit Function Theorem |
|
|
475 | (2) |
|
|
477 | (4) |
|
|
481 | (2) |
|
B Derivations and Proofs of Technical Results |
|
|
483 | (22) |
|
B.1 Separation Theorems, Farkas Lemma and Supergradient |
|
|
483 | (3) |
|
B.2 Proof of the Michel Theorem |
|
|
486 | (1) |
|
B.2.1 Augmented and Truncated Problem |
|
|
487 | (1) |
|
B.2.2 Optimal Solution of Problem (B.8) |
|
|
487 | (1) |
|
B.2.3 Necessary Conditions for Problem (B.8) |
|
|
488 | (1) |
|
B.2.4 Limit of Solutions for Increasing Time Sequence |
|
|
489 | (2) |
|
B.3 Proof of the Transversality Condition in Proposition 3.74 |
|
|
491 | (1) |
|
B.4 The Infinite Horizon Transversality Condition Revisited |
|
|
492 | (2) |
|
B.5 Monotonicity of the Solution Path |
|
|
494 | (2) |
|
B.6 Admissible and Quasi-Admissible Directions |
|
|
496 | (2) |
|
B.7 Proof of the Envelope Theorem |
|
|
498 | (1) |
|
B.8 The Dimension of the Stable Manifold |
|
|
499 | (3) |
|
B.9 Asymptotic Boundary Condition |
|
|
502 | (1) |
|
|
502 | (1) |
|
|
503 | (2) |
References |
|
505 | (26) |
Glossary |
|
531 | (4) |
Index |
|
535 | (10) |
Author Index |
|
545 | |