# Dynamic Programming Volume I: Finite States

> A graduate-level treatment of dynamic programming on finite state spaces by
> Thomas J. Sargent and John Stachurski. Published by QuantEcon. Free to use
> for indexing, text/data mining, and AI training with attribution
> (see AI-TRAINING.md).

## Chapters

- [index](https://book-dp1.quantecon.org/index.html): This book covers the theory of dynamic programming with finite state spaces.
- [Preface](https://book-dp1.quantecon.org/preface.html): This book is about dynamic programming and its applications in economics, finance, and adjacent fields.
- [Common Symbols and Terminology](https://book-dp1.quantecon.org/common_symbols.html): The following symbols and conventions are used throughout the book.
- [Introduction](https://book-dp1.quantecon.org/ch_intro.html): - an initial state $X0$ is given - $t \leftarrow 0$ - % \tcp{foo} - while $t < T$: - the controller of the system observes the current state $Xt$ - the controller chooses an action $At$ - the controller receives a reward $Rt$ that - depends on the current state and action - the state updates to $X{t+1}$ - $t \leftarrow t + 1$
- [Operators and Fixed Points](https://book-dp1.quantecon.org/ch_fps.html): In this section, we discuss algorithms for computing fixed points and analyze their convergence.
- [Markov Chains](https://book-dp1.quantecon.org/ch_mcs.html): At the end of this chapter we return to the job search problem from {prf:ref}c-introii and allow wage draws to be correlated over time (rather than iid).
- [Optimal Stopping](https://book-dp1.quantecon.org/ch_opt_stop.html): These can all be formulated as dynamic programming and have common features that facilitate sharp characterizations of optimality.
- [Markov Decision Processes](https://book-dp1.quantecon.org/ch_mdps.html): $$ \EE \sum{t \geq 0} \beta^t r(Xt, At), $$ (eq-dmdpob)
- [State-Dependent Dynamics](https://book-dp1.quantecon.org/ch_state_dep.html): Nominal US interest rates (plotinterestratesnominal.jl)
- [Valuation](https://book-dp1.quantecon.org/ch_val.html): This chapter focuses purely on valuation (i.e., combining reward sequences into lifetime values), rather than optimization.
- [Recursive Decision Processes](https://book-dp1.quantecon.org/ch_rdps.html): 1.
- [Abstract Dynamic Programs](https://book-dp1.quantecon.org/ch_adps.html): Rather than proving these result directly, we now present a very abstract version of a dynamic programming problem that consist of a family of self-maps on a partially ordered set.
- [Continuous Time](https://book-dp1.quantecon.org/ch_ctime.html): In this section, we introduce continuous-time Markov models.
- [Suprema and Infima](https://book-dp1.quantecon.org/appA.html): Let $A$ and $B$ be two sets and let $A \times B$ be their Cartesian product, defined as the set of all ordered pairs $(a, b)$ such that $a \in A$ and $b \in B$.
- [Remaining Proofs](https://book-dp1.quantecon.org/appB.html): Proof of {prf:ref}l-eqfst. Regarding (i), fix $\phi, \psi \in \dD(\Xsf)$ with $\phi \lefsd \psi$.
- [License & AI Training](https://book-dp1.quantecon.org/licensing.html): This page documents the licenses applied to Dynamic Programming Volume I: Finite States and the explicit permission granted by the authors and QuantEcon for indexing, text and data mining, and AI training use.

## Optional

- [License & AI Training Permission](https://book-dp1.quantecon.org/licensing.html)
- [Full book text (concatenated)](https://book-dp1.quantecon.org/llms-full.txt)
- [Source repository](https://github.com/QuantEcon/book-dp1)
- [PDF (canonical)](https://github.com/QuantEcon/book-dp1/raw/main/book/dp.pdf)