Foreword

It is a special privilege for me to write a few words for this book, both because of its subject and because of its intellectual connection with Dimitri P. Bertsekas.

I came to know Dimitri while I was a Ph.D. student at the Royal Institute of Technology in Sweden, as a reader of one of his many books. Later, I worked closely with him at Arizona State University, where he was my mentor and research adviser. Over time, however, our relationship became much more than an academic one. To me, Dimitri was like a loving father. He was extraordinarily generous with his time, his ideas, and his encouragement. He seemed to know almost everything, yet he remained remarkably humble and open-minded. He was always ready to listen, to reconsider an argument, and to help at the slightest sign that I was in difficulty. From him I learned much about mathematics and research, but also something more important: a lasting faith in the best qualities that scholarship and human character can represent.

Dynamic programming occupied a very important place in Dimitri’s unusually broad scientific work. His contributions to the subject extended over half a century, from his Ph.D. thesis and his first book, Dynamic Programming and Stochastic Control, published in 1976, to his later work on reinforcement learning and his monograph Abstract Dynamic Programming. A characteristic of Dimitri’s thinking across this body of work was the search for the right conceptual structure. He was never satisfied merely to prove a theorem or devise an algorithm. He wanted to understand why an idea worked, how far it could be extended, how apparently different problems were connected, and how all of this could be explained with clarity. For this reason, he took great pride in his 1976 book, particularly in its unified treatment of a wide range of problems, an emphasis it shared with Richard E. Bellman’s influential book Dynamic Programming.

This book by Thomas J. Sargent and John Stachurski continues very much in that spirit. More specifically, its general theory connects directly to Dimitri’s research on abstract dynamic programming, which began in 1975, while taking the abstraction in new directions. In particular, it places order-preserving policy operators at the center of the theory and develops an order-theoretic framework that reaches well beyond the classical theory based on contraction mappings. The result is a unified treatment that ranges from standard Markov decision processes to recursive preferences, risk-sensitive and robust control, distributional dynamic programming, and other models that would otherwise appear quite different from one another.

I think Dimitri would have appreciated this book. He valued abstraction when it illuminated rather than obscured, and theory when it brought seemingly separate ideas together while remaining connected to useful models and algorithms. He also cared deeply about exposition. For him, writing was part of the creative process of research: finding the clearest formulation was often inseparable from finding the right idea.^[1] This book shares that aspiration.

Dimitri’s passing leaves an immense absence for those of us who knew and loved him. Yet his influence remains remarkably alive. It is seen in his students, collaborators, books, and ideas, and in new work that takes those ideas in directions he helped make possible. This volume is one such continuation. I am grateful to its authors for carrying forward a line of thought that meant so much to Dimitri, and I hope that readers will find in these pages some of the same intellectual excitement that dynamic programming brought him throughout his life.

Yuchao Li
Arizona State University
September 2026