Close

Complete Captcha

• 3 years - 40 posts - Latest - RSS
• Thread: 117 Goods vs 30 No Goods

1. Economist
   217b
   

   Master student here.

   I am looking for any resources for the intuition on how to use DP and solved examples (not code).

   I more or less got the mathematical part like blackwell conditions, principle of optimality etc, but I can't understand how to apply it (e.g. what state and control variables to use, what constraints matter, how many Bellman equations, are needed to describe the equilibrium , what value functions are there and so on).

   Most books/notes I've seen just have one extremaly simple example with 1 state, 1 control for the growth model, which is obvious. However, extending this to many variables and functions was rather hard for me.

   3 years ago # QUOTE 1 Volod 2 Vlad !

2. Economist
   5ae9
   

   nobody knows.

   3 years ago # QUOTE 13 Volod 0 Vlad !

3. Economist
   d3df
   

   Puterman's Markov Decision Processes book as a lot of examples in the early chapters. See also Powell's Approximate Dynamic Programming book. There are many examples in there.

   3 years ago # QUOTE 3 Volod 0 Vlad !

4. Economist
   217b
   

   Puterman's Markov Decision Processes book as a lot of examples in the early chapters. See also Powell's Approximate Dynamic Programming book. There are many examples in there.

   Thanks for the suggestions.

   3 years ago # QUOTE 0 Volod 0 Vlad !

5. Economist
   e8bc
   

   Nobody really knows. Macroeconomists just repeat whatever their teachers taught them. The moment you ask a weird question to them, they panic.

   3 years ago # QUOTE 23 Volod 1 Vlad !

6. Economist
   9e1c
   

   Nobody really knows. Macroeconomists just repeat whatever their teachers taught them. The moment you ask a weird question to them, they panic.

   You should never ask a weird question to them!

   3 years ago # QUOTE 5 Volod 0 Vlad !

7. Economist
   e8bc
   

   By weird I just meant going slightly off the script...

   Nobody really knows. Macroeconomists just repeat whatever their teachers taught them. The moment you ask a weird question to them, they panic.

   You should never ask a weird question to them!

   3 years ago # QUOTE 6 Volod 1 Vlad !

8. Economist
   27d8
   

   Check out Chris Carroll’s lecture notes.

   3 years ago # QUOTE 1 Volod 0 Vlad !

9. Economist
   217b
   

   Check out Chris Carroll’s lecture notes.

   Thanks, looking into it.

   3 years ago # QUOTE 0 Volod 0 Vlad !

10. Economist
    6299
    

    Adda Cooper is the best book out there for starters. You can find a free soft copy on the internet too.

    3 years ago # QUOTE 4 Volod 2 Vlad !

11. Economist
    db02
    

    Christian traeger and Larry karp have some very good lecture notes/book chapters.

    3 years ago # QUOTE 0 Volod 1 Vlad !

12. Economist
    952b
    

    tim roughgarden also recently published part three of his algorithms illuminated books. i have not read that part yet, but the series is more approachable than other algorithm book classic.s

    3 years ago # QUOTE 0 Volod 0 Vlad !

13. Economist
    ada2
    

    Nobody really knows. Macroeconomists just repeat whatever their teachers taught them. The moment you ask a weird question to them, they panic.

    Actually true, lol.

    2 years ago # QUOTE 5 Volod 0 Vlad !

14. Economist
    9e60
    

    I took several PhD math courses before I took phD macro. Dynamic programming was harder/less intuitive than anything I had encountered before.

    I think the intuition in this field is just hard to grasp for students, because it's basically invented by brilliant economists making up math tools on their way to solve econ problems. You gotta have a few years of experience in solving those econ problems before you can backwards-induct why dynamic programming is made the way it is.

    2 years ago # QUOTE 0 Volod 9 Vlad !

15. Economist
    5b33
    

    I took several PhD math courses before I took phD macro. Dynamic programming was harder/less intuitive than anything I had encountered before.
    I think the intuition in this field is just hard to grasp for students, because it's basically invented by brilliant economists making up math tools on their way to solve econ problems. You gotta have a few years of experience in solving those econ problems before you can backwards-induct why dynamic programming is made the way it is.

    Actually invented by Richard Bellmann, a mathematician.

    2 years ago # QUOTE 13 Volod 0 Vlad !

16. Economist
    d764
    

    I took several PhD math courses before I took phD macro. Dynamic programming was harder/less intuitive than anything I had encountered before.
    I think the intuition in this field is just hard to grasp for students, because it's basically invented by brilliant economists making up math tools on their way to solve econ problems. You gotta have a few years of experience in solving those econ problems before you can backwards-induct why dynamic programming is made the way it is.

    Were those PhD math courses offered by the University of Phoenix?

    2 years ago # QUOTE 8 Volod 2 Vlad !

17. Economist
    6635
    

    Puterman's Markov Decision Processes book as a lot of examples in the early chapters. See also Powell's Approximate Dynamic Programming book. There are many examples in there.

    This. The Dynamic programming PhD math course I took used Putterman, and it's a great reference book. Bertsekas' Dynamic Programming and Optimal Control books are good as well, and I personally found Kallenberg's notes (https://www.math.leidenuniv.nl/~kallenberg/Lecture-notes-MDP.pdf) very useful.

    2 years ago # QUOTE 0 Volod 0 Vlad !

18. Economist
    cf62
    

    Puterman's Markov Decision Processes book as a lot of examples in the early chapters. See also Powell's Approximate Dynamic Programming book. There are many examples in there.

    Puterman Strong recommend

    2 years ago # QUOTE 0 Volod 0 Vlad !

19. Economist
    9794
    

    If you want to solve an abstract dynamic decision problem, in principle you need to know all the primitives, and everything you’ve done and observed in the past.

    Think of all of this as your "state" variables.

    Of course, very many of these you won’t need to keep track of explicitly: for example, if your objective is just a present discounted sum of period utilities, then your period utility and discount factor will remain fixed, so you can just keep them in the background without making explicit the dependence of your value function on these objects.

    Ok, now that you’ve stopped keeping track of all the things that remain fixed, you have the stuff that changes over time (e.g. the history of actions and things you’ve observed).

    Typically, your problem will have a special structure that will allow you to reduce this data into a few summary statistics. For example, if your payoff in each period depends only on your action and an unknown persistent state of the world (over which you have a prior), then the history of actions and observations will allow you to compute a posterior over that state which is a sufficient statistic for computing the optimal thing to do next.

    These sufficient statistics are what you want to keep as your state variables. You want this set of state variables to be as small and simple as possible, without throwing out data you actually need to compute the optimal action in each period.

    2 years ago # QUOTE 6 Volod 2 Vlad !

20. Economist
    4b12
    

    If your not understanding the mechanism of dynamic programming, you should not pursue macro.

    2 years ago # QUOTE 3 Volod 0 Vlad !

Reply »