8681: Having tried neural networks before pretraining, and probably even before backpropogation, is equivalent to not having tried modern neural networks at all.
Deep Learning applications in economics?

Economic forecasting in any form, machine learning or not, gets zero traction in top journals. The reason is that there are very few interesting questions that can be forecasted with any accuracy no matter what method is used, for equilibrium reasons that economists discuss all the time. I could see some use for machine learning is situations like Fed green books, perhaps, or maybe for certain types of demand estimation (e.g., what parameters are useful in including in the model).

Economic forecasting in any form, machine learning or not, gets zero traction in top journals. The reason is that there are very few interesting questions that can be forecasted with any accuracy no matter what method is used, for equilibrium reasons that economists discuss all the time. I could see some use for machine learning is situations like Fed green books, perhaps, or maybe for certain types of demand estimation (e.g., what parameters are useful in including in the model).
I dont think economists have spent enough time with machine learning models to know that you cant forecast well. I think a bayesian Lstm with added information via global economic data could perform better than whats currently out there.

You can build a bunch of models that share an assumption, then run DEEP LEARNING on them, then let the deep learning generate a new model on the basis of the learning. When that new model is also compatible with the assumptions, you can then take this as proof that the assumptions are true and that we should organise society to fit with the assumption.
 economist 
I see at least two issues:
(I) ECONOMISTS WHO JUST BLINDLY IMPORT ML PACKAGES: ML and deep learning are all the rage these days, and consequently there are lots of well developed packages (e.g. Tensorflow, scikitlearn, etc) that give you canned routines to implement them. In the last two to three years, there indeed have been tons of papers that take on an economics motivated question and some economics data, and try to run on one of these canned procedures. These types of papers fall into two camps: (a) With these variables, the paper claims it can improve forecasting accuracy of some key variable; or (b) The ML procedure outputs a fitted value, and the economist falls back to using OLS to make causality claims. What are the problems? As many posters have mentioned above, the top journals simply don't care about (a). Forecasting accuracy, whether it be insample or outofsample, don't matter to top journal editors. Is there any hope for application (b)? I think not. Using a fitted ML output as a regressor in an OLS type setting is exactly a phacking exercise because you are reusing the data twice. In OLS, it is akin to first regress on 10 variables, pick out the top 3 that have stars, drop the other 7 insignificant variables, and then run the OLS again with just those 3. The resulting tstats out of these "look ahead" regressions are completely not reliable. In all, application (a) gets discarded by top journals, and application (b) is just outright questionable or even fraudulent.
(II) YOUR AVERAGE APPLIED ECONOMIST DOES NOT CARE / HAS NO ABILITY TO REFINE ON ML METHODS. In light of (I), can your applied economist  armed with, of course, a proprietary dataset  design new ML methods to tackle the economic question they want to answer? I claim that with the current incentive structure in academic hiring and promotion, the answer is no. Put it bluntly, your average reg monkey struggles to even understand what are the deep theoretical econometric results of even the OLS estimator. How do you expect to understand even a refinement of OLS? For example, to understand the theoretical properties of Lasso or the ElasticNet, one needs a pretty good understanding of statistics in Hilbert spaces. Your average applied economist chose to do applied economics back in grad school precisely because they want to AVOID dealing with fancy maths. You may argue  ok, how about we hire econometricians to study these problems? Sir, have you seen what the econometrics job market looks like? It is dismal. Moreover, the problems that econometricians study are usually quite orthogonal to what interests the CS crowd anyway. You may then argue  ok, how about we economists collaborate with CS academics? Jeez, given that economists are supposed to be wellversed in understanding incentives, you have to ask yourself what is the incentive of a productive CS researcher in spending time solving an economist's problem? This is especially when fields like biostatistics, medical imaging and robotics are far more receptive to their ideas than the conservative (or even outright closeted) intellectual culture of economics.
Given (I) and (II), I seriously don't see any fruit applications of ML, or advancements of ML will come from economists.