Future of Stata « Economics Job Market Rumors

8 months - 57 posts - Latest - RSS
Thread: 100 Goods vs 41 No Goods

Economics Job Market Rumors » Economics » Software and Programming for Research

Future of Stata

Economist
39bc

seriously, this language isn't going anywhere. The developers have the wrong priorities.
Things they focus on:
- time series
- DSGE models

Things they should focus on:
- parallelization
- fast egen
- do file editor

But I have low hopes. The whole company is full of lemons. Did you for example know that inlist(y,x1,x2...) only supports checking for up to 10 arguments? How come all functions I use (hdfereg, binscatter, gtools, ftools, parallel,............) are written by users? What do the people do at Stata all day?

8 months ago # QUOTE 23 Good 2 No Good !
Economist
c4c8

who are you?

8 months ago # QUOTE 3 Good 3 No Good !
Economist
4fc7

As long as the profession keeps producing coding-deficient economists, they'll do just fine.

8 months ago # QUOTE 4 Good 3 No Good !
Economist
e53a

For decades, Stata’s main market segment has been the academic market. Under the assumption that medium to reasons medium size data sets are being used, they concentrated in weaponizing estimators, test statistics, etc. published in major journals in the field.

For the past 5 years, Econometrics has turned itself back to model selection (but in high dimensions). This is an old topic, but there is a tweak: they didn’t have powerful computers and terabytes worth of dada back in the 50s.

The same way every single microeconometrics mode was ‘re-discovered’ under partial identification, every single model in both Microeconometrics and Time Series (which per EJMR folklore is dead) will be ‘re-discovered’ under high-dimensionality and big data.

If Stata wants to survive, it must go back to revise its original assumptions about data size and data handling.

seriously, this language isn't going anywhere. The developers have the wrong priorities.
Things they focus on:
- time series
- DSGE models
Things they should focus on:
- parallelization
- fast egen
- do file editor
But I have low hopes. The whole company is full of lemons. Did you for example know that inlist(y,x1,x2...) only supports checking for up to 10 arguments? How come all functions I use (hdfereg, binscatter, gtools, ftools, parallel,............) are written by users? What do the people do at Stata all day?

8 months ago # QUOTE 1 Good 0 No Good !
Economist
b78c

Stata doesn't have a future. It is stone age software. Statacorp didn't really improve the software in any useful way for the last 15 years.

8 months ago # QUOTE 2 Good 2 No Good !
Economist
eed0

So what is the future? R, perhaps?

8 months ago # QUOTE 0 Good 2 No Good !
Economist
0b3a

These days, as another poster mentioned, Stata's strength is in its user-written add ons, which are largely written by a group of older people brought up on Stata. These cohorts are slowly being replaced with cohorts using R and other free software.

Stata is dying a slow death.

It's sad because I love lighting up a satisfying Stata in the morning, but it's just not fashionable any more.

8 months ago # QUOTE 7 Good 0 No Good !
Economist
4c01

Same as SAS. It will become a niche products for consulting where people are too used to old software. The license prices will keep increasing to reflect the cream skimming

8 months ago # QUOTE 4 Good 0 No Good !
Economist
b26d

Don't forget the induced data file incompatibilities between versions, so that I have to use StatTransfer every week now. And the licensing model.

I am finishing my existing projects in Stata and starting new things in R. Hopefully I can build competency slowly in the next few years.

8 months ago # QUOTE 1 Good 0 No Good !
Economist
ff29

So what is the future? R, perhaps?

R, Python and Julia not yet clear which will dominate or even whether one language will dominant or we will move more to ecosystems made up of a mix of languages, I think this is more likely.

8 months ago # QUOTE 3 Good 0 No Good !
Economist
b86d

R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.

Stata is fine for handling large datasets, that's where a lot of development effort went recently.

But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.

Stata + Python, forget R.

8 months ago # QUOTE 5 Good 5 No Good !
Economist
529d

I'd say the last good thing they did was strL in v13. they changed the format for v14, so I upgraded, but I see no point in v15.

good point is that most good functions are user-written. I'm not sure why they don't incorporate f/gtools - merge/collapse are horrendously slow. That said, those functions are memory-thirsty so maybe they are trying to trade that off vs. speed.

But yes, it will die a slow death. And i say this as someone who uses it every day and has no plans to switch to R.

8 months ago # QUOTE 1 Good 1 No Good !
Economist
9754

Lots of economists want to be glorified data scientists. These guys don't like S.T.A.T.A. that much. Real economists don't mind that S.T.A.T.A. can't quickly loop through a p-hacking routine on the 20 billion tweets that they web-scraped. They want to run the estimator they need without having to spend a week "coding."

8 months ago # QUOTE 7 Good 1 No Good !
Economist
cc98

First, Stats is going nowhere. It is even more popular and more used compared to 17 years ago say, if anything Stata is on the rise.

Second, OP is totally right that the Stata developers, executives, and owners are complete douchebags, and they have their priorities all messed up.

a) Selling Stata MP as something special and for "special" price, after we have 4 cores even on our phones, for 5-7 years now? Seriously?

b) Developing the wrong tools, Bayesian bulsheit and DSGE models, when we need better ability to handle big data and better and faster egen, merge, etc...

c) Moving in this sheity R direction where everything is done by a special package...

8 months ago # QUOTE 6 Good 0 No Good !
Economist
8e53

R is bs.

Stata STRONG

8 months ago # QUOTE 0 Good 2 No Good !
Economist
529d

does anyone know how much of Stata actually benefits from multiprocessors?

obviously estimation speeds up, and I think I have seen it spring to MP mode for sort, but does merge/joinby/collapse and other data manipulation routines benefit AT ALL?

i find myself having to split datafiles into dozens of pieces and execute parallel jobs on the server. I can get 10-100x speedup in performance doing this, whereas if Stata chugs along singlethreadedly it takes for-evah.

8 months ago # QUOTE 0 Good 0 No Good !
Economist
cb3b

Same as SAS. It will become a niche products for consulting where people are too used to old software. The license prices will keep increasing to reflect the cream skimming

Lumping SAS's and Stata's futures together makes as much sense as lumping together your local sandwich shop's future with that of McDonald's. Not entirely uncorrelated, but totally different beasts.

8 months ago # QUOTE 0 Good 0 No Good !
Economist
bb5c

R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.

I gave this a TRADE for use of the term bigly.

8 months ago # QUOTE 1 Good 2 No Good !
Economist
39bc

Yes the pricing. Who thinks it makes sense to base the pay on the number of variables and cores you use?

529d: Try parallel. It does that for you automatically. Also use gtools/ftools wherever you can.

8 months ago # QUOTE 0 Good 0 No Good !
Economist
529d

Yes the pricing. Who thinks it makes sense to base the pay on the number of variables and cores you use?
529d: Try parallel. It does that for you automatically. Also use gtools/ftools wherever you can.

hmm, I am always skeptical of these supposed 'speedup' utilities. gtools for examples is a shocking memory hog (ftools somewhat less so). does 'parallel' have no downsides, really?

8 months ago # QUOTE 0 Good 0 No Good !

12 3 Next »

Recently Announced

Upcoming

Economics

Job Market

Off Topic

Job Market Wiki

Job Market Candidates

Updates

Future of Stata

Reply »