Of course you need memory to hold the data in memory simultaneously
Future of Stata
-
R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.No, Stata is garbage for handling large datasets, and it has made virtually no progress on this front in at least five years. Collapses, merges, reshapes, etc all take way too long compared to R. User-written functions like gtools help Stata become more usable, but none of those efforts have been endorsed or embraced by Statacorp.
-
R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.No, Stata is garbage for handling large datasets, and it has made virtually no progress on this front in at least five years. Collapses, merges, reshapes, etc all take way too long compared to R. User-written functions like gtools help Stata become more usable, but none of those efforts have been endorsed or embraced by Statacorp.
They instead embrace packages like import Fred:
https://www.stata.com/new-in-stata/import-fred/ -
R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.No, Stata is garbage for handling large datasets, and it has made virtually no progress on this front in at least five years. Collapses, merges, reshapes, etc all take way too long compared to R. User-written functions like gtools help Stata become more usable, but none of those efforts have been endorsed or embraced by Statacorp.
They instead embrace packages like import Fred:
https://www.stata.com/new-in-stata/import-fred/which of course is just a repackaged version of the freduse program that few people cared about. statacorp is a bad company and they deserve to fail.
-
R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.No, Stata is garbage for handling large datasets, and it has made virtually no progress on this front in at least five years. Collapses, merges, reshapes, etc all take way too long compared to R. User-written functions like gtools help Stata become more usable, but none of those efforts have been endorsed or embraced by Statacorp.
merge is still single-threaded (because it does disk I/O). not sure how they will speed it up, ever. is R materially faster at merging? if so then I may switch. still chopping up my files for merging etc.
gtools is giod but a real memory hog.
-
R is inefficient relative to Stata, in terms of memory management, syntax, and conflict management between user packages.
Stata is fine for handling large datasets, that's where a lot of development effort went recently.
But optimization still sucks bigly, and indeed one has to use user-written code a bit too often.
Stata + Python, forget R.No, Stata is garbage for handling large datasets, and it has made virtually no progress on this front in at least five years. Collapses, merges, reshapes, etc all take way too long compared to R. User-written functions like gtools help Stata become more usable, but none of those efforts have been endorsed or embraced by Statacorp.
merge is still single-threaded (because it does disk I/O). not sure how they will speed it up, ever. is R materially faster at merging? if so then I may switch. still chopping up my files for merging etc.
gtools is giod but a real memory hog.Merging, like most other things, is a lot faster in R: https://github.com/matthieugomez/benchmark-stata-r
-
OP and most of his commentators are clearly dumb computer-scientists who never came to study economics.
The fact that Stata has many technical limits _now_ does not imply it will lose its leadership anytime in the future: it is a QWERTY story, the market benefits from using a widely diffused software, whose language is now well-known.
To beat Stata leadership in the market the alternative software does not have to rely on technical improvements. It may even be less efficient or lack some (possibly outdated and no-longer used) estimators.
What the true competitor would need is a conceptually easier language, for which the learning curve is way faster.Today (and, I believe, always) mass consumers (the
academic community'' is now much larger than before) value time more than technical features.
A nice example is the diffusion of Lyx, which is going to dominate over standard MikTex softwares very soon. Alas, it is a crappy software, no efficiency in coding, full of compatibility issues... yet the learning curve is ... IMMEDIATE! -
OP and most of his commentators are clearly dumb computer-scientists who never came to study economics.
The fact that Stata has many technical limits _now_ does not imply it will lose its leadership anytime in the future: it is a QWERTY story, the market benefits from using a widely diffused software, whose language is now well-known.
To beat Stata leadership in the market the alternative software does not have to rely on technical improvements. It may even be less efficient or lack some (possibly outdated and no-longer used) estimators.
What the true competitor would need is a conceptually easier language, for which the learning curve is way faster.
Today (and, I believe, always) mass consumers (theacademic community'' is now much larger than before) value time more than technical features.
A nice example is the diffusion of Lyx, which is going to dominate over standard MikTex softwares very soon. Alas, it is a crappy software, no efficiency in coding, full of compatibility issues... yet the learning curve is ... IMMEDIATE!False premise. The R community is many times bigger than the Stata community. Stata has more estimators out of the box, but the development community for R is far larger than than Stata's. And lyx sucks.