Just use Hamiltonians
Struggling to really understand dynamic programming
-
If you want to solve an abstract dynamic decision problem, in principle you need to know all the primitives, and everything you’ve done and observed in the past.
Think of all of this as your "state" variables.
Of course, very many of these you won’t need to keep track of explicitly: for example, if your objective is just a present discounted sum of period utilities, then your period utility and discount factor will remain fixed, so you can just keep them in the background without making explicit the dependence of your value function on these objects.
Ok, now that you’ve stopped keeping track of all the things that remain fixed, you have the stuff that changes over time (e.g. the history of actions and things you’ve observed).
Typically, your problem will have a special structure that will allow you to reduce this data into a few summary statistics. For example, if your payoff in each period depends only on your action and an unknown persistent state of the world (over which you have a prior), then the history of actions and observations will allow you to compute a posterior over that state which is a sufficient statistic for computing the optimal thing to do next.
These sufficient statistics are what you want to keep as your state variables. You want this set of state variables to be as small and simple as possible, without throwing out data you actually need to compute the optimal action in each period.Gibberish. Sufficient statistics is not the right way to think about state variables at all
-
lol @ econ people in this thread thinking they know maths. OP, if you don't have a masters in maths or heavily mathematical statistics, don't bother. Just copy what other people have done, make up a very little twist and publish in econ journals. You need 3 years at least of investment in pure maths to understand dynamic programming because it connects so many branches. In fact to understand the topic at state of the art you have to be both a probability theory and PDE theory grandmaster which is rare among maths or statistics people and impossible to attain for economists. It is what it is
-
lol @ econ people in this thread thinking they know maths. OP, if you don't have a masters in maths or heavily mathematical statistics, don't bother. Just copy what other people have done, make up a very little twist and publish in econ journals. You need 3 years at least of investment in pure maths to understand dynamic programming because it connects so many branches. In fact to understand the topic at state of the art you have to be both a probability theory and PDE theory grandmaster which is rare among maths or statistics people and impossible to attain for economists. It is what it is
Depends. Do you want to use dynamic programming, or solve HJB existence problems?
-
Gibberish. Sufficient statistics is not the right way to think about state variables at all
Completely agreed. State variables are just the variables that define the system in period t. They cannot be chosen in the current period, they just represent the state. A sufficient statistic in DP is a function of a random variable that is sufficient to tell you what you need to evaluate the next iteration of the Bellman operator. A sufficient statistic is absolutely not the same thing as a state variable. It is a statistic, not a variable. The people downvoting you have no clue about dynamic programming.
-
lol @ econ people in this thread thinking they know maths. OP, if you don't have a masters in maths or heavily mathematical statistics, don't bother. Just copy what other people have done, make up a very little twist and publish in econ journals. You need 3 years at least of investment in pure maths to understand dynamic programming because it connects so many branches. In fact to understand the topic at state of the art you have to be both a probability theory and PDE theory grandmaster which is rare among maths or statistics people and impossible to attain for economists. It is what it is
Lol, what a poser. Good one bro
-
Gibberish. Sufficient statistics is not the right way to think about state variables at all
Completely agreed. State variables are just the variables that define the system in period t. They cannot be chosen in the current period, they just represent the state. A sufficient statistic in DP is a function of a random variable that is sufficient to tell you what you need to evaluate the next iteration of the Bellman operator. A sufficient statistic is absolutely not the same thing as a state variable. It is a statistic, not a variable. The people downvoting you have no clue about dynamic programming.
I don’t see why you think this is such a bad interpretation of a state variable. The state captures everything that is relevant for the agent’s decision. That seems very similar to a sufficient statistic.
-
There is nothing to understand per se, as an economist you will be using dynamic programming like you use calculus -- as a tool to derive equations that represent optimal (intertemporal) behavior. It is time economists stop mathturbating and treat maths like physicist do, as a means to an end.
-
There is nothing to understand per se, as an economist you will be using dynamic programming like you use calculus -- as a tool to derive equations that represent optimal (intertemporal) behavior. It is time economists stop mathturbating and treat maths like physicist do, as a means to an end.
economists are a special breed who are not smart enough to do math but think they are. especially Americans.
-
Gibberish. Sufficient statistics is not the right way to think about state variables at all
Completely agreed. State variables are just the variables that define the system in period t. They cannot be chosen in the current period, they just represent the state. A sufficient statistic in DP is a function of a random variable that is sufficient to tell you what you need to evaluate the next iteration of the Bellman operator. A sufficient statistic is absolutely not the same thing as a state variable. It is a statistic, not a variable. The people downvoting you have no clue about dynamic programming.
Excuse me but a statistic is a variable, to be precise it is a function of a random variable, so when you write that "it is a statistic, not a variable" you are writing non-sense.
-
If your not understanding the mechanism of dynamic programming, you should not pursue macro.
this.
The actual reason is macro today= applied dynamic programming. q.e.d.Absolutely not. In fact in pretty much any situation you can just do a Lagrangian with as many multipliers as there are periods, take derivatives, and be done with it. This yields the same answers as the DP approach, and from memory it is the only theorem that is actually important in SLP.
-
There is nothing to understand per se, as an economist you will be using dynamic programming like you use calculus -- as a tool to derive equations that represent optimal (intertemporal) behavior. It is time economists stop mathturbating and treat maths like physicist do, as a means to an end.
economists are a special breed who are not smart enough to do math but think they are. especially Americans.
stop attacking my friends on ejrm