Why doesn't R ever get any love? « Economics Job Market Rumors

4 months - 61 posts - Latest - RSS
Thread: 81 Goods vs 29 No Goods

Economics Job Market Rumors » Off Topic » Off Topic

Why doesn't R ever get any love?

Economist
dd65

It's an awesome language for non programming peons. It's absolutely awesome for statistics, it's relatively easy to use, there's a package for everything. Why do people like to h@te on R?

I think overall R is well liked, compared to Stata and Matlab at least, because it is free (as in freedom), as a huge amount of libraries for all statistical things and can even to an extent serve as a general purpose scripting language like Python. R is too slow for serious numerical work however. For the latter C/C++ or Fortran are mandatory.

EJMR always talks about speed because they have no idea what actually differentiates languages.

Idk what people here do but I've never had issues with speed

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

4 months ago # QUOTE 3 Volod 0 Vlad !
Economist
232d

I love R. Nnet suuuuuuuuuuucks

There is usually (definitely not always) some package that does what you want to do, but it may be poorly implemented, abandoned, not play well with packages by other people you also need, etc. Even pretty big packages like URCA I have run into numerical errors that would never happen in SAS or Stata.
Syntax is a mess because of different packages and it is very difficult to switch between tidyverse/data.table/whatever.
SAS dominates it for handling large datasets smoothly.

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
39fe

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

My luck: used a comma instead of a semicolon. Need to run again.

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
232d

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

The answer is “do it in openmp” which plays nice with R but, uh, that’s not a great answer. R handles parallelization but…. Use c++

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
cf89

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

I'm pretty sure professionally written Python libraries like Tensorflow Probability run faster than some jerry-rigged C++ code written by some econ amateur.

4 months ago # QUOTE 2 Volod 3 Vlad !
Economist
dd65

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

The answer is “do it in openmp” which plays nice with R but, uh, that’s not a great answer. R handles parallelization but…. Use c++

Parallelization in scripting languages is awkward because of technical considerations (global interpreter lock and the cost of spawning the slave threads). In addition, if the problem is sequential (Markov) parallelization may not be possible at all.

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
699f

or t(matrix(1:9, 3, 3)). Also what kind of projects are you working on that inputting a small matrix is the bottleneck? I don't think anything matches dplyr and ggplot2 in ease of use for data manipulation and plotting.

A=[ 1 2 3; 4 5 6; 7 8 9]

A = matrix(
c(1, 2, 3, 4, 5, 6, 7, 8, 9),
nrow = 3,
ncol = 3,
byrow = TRUE
)
Why would any not like that?

This is fine. It's programming.

rbind(1:3,4:6,7:9)
Does the same thing. Don't blame your ignorance on R.

4 months ago # QUOTE 2 Volod 1 Vlad !
Economist
dd65

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

I'm pretty sure professionally written Python libraries like Tensorflow Probability run faster than some jerry-rigged C++ code written by some econ amateur.

We are talking structural work here not cookie cutter machine learning. It is not difficult to write C code that is faster than standard R or Python libraries by optimizing array allocations and removing checks which are known to be satisfied in 5e context of a particular application. Doing so can slash the running time even of C++ STL functions by a factor of 10 with minimal effort. But what would a reg monkey know.

4 months ago # QUOTE 2 Volod 2 Vlad !
Economist
232d

At a certain point ggplot is ms paint- I’m feeding it a list of values hoping it renders. At that point, excel is better

or t(matrix(1:9, 3, 3)). Also what kind of projects are you working on that inputting a small matrix is the bottleneck? I don't think anything matches dplyr and ggplot2 in ease of use for data manipulation and plotting. >

4 months ago # QUOTE 1 Volod 0 Vlad !
Economist
cf89

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

I'm pretty sure professionally written Python libraries like Tensorflow Probability run faster than some jerry-rigged C++ code written by some econ amateur.

We are talking structural work here not cookie cutter machine learning. It is not difficult to write C code that is faster than standard R or Python libraries by optimizing array allocations and removing checks which are known to be satisfied in 5e context of a particular application. Doing so can slash the running time even of C++ STL functions by a factor of 10 with minimal effort. But what would a reg monkey know.

Bollocks. Really fast code would need to be massively parallelized, and optimized to run on CUDA. Google and the rest of FANG has a thousand CS PhDs working on this stuff, optimizing libraries that are actually used in production. You're delusional if you think your amateur C code can beat that.

4 months ago # QUOTE 0 Volod 1 Vlad !
Economist
39fe

At a certain point ggplot is ms paint- I’m feeding it a list of values hoping it renders. At that point, excel is better

or t(matrix(1:9, 3, 3)). Also what kind of projects are you working on that inputting a small matrix is the bottleneck? I don't think anything matches dplyr and ggplot2 in ease of use for data manipulation and plotting. >

If you want your plots to look like Excel plots, Excel is better

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
232d

Google people have operational constraints that I don’t. They could write faster code than I could if they tried to solve my problem, but the problem they are trying to solve isn’t really MY problem. So my code is faster for my problem. It’s not that hard to understand.

We are talking structural work here not cookie cutter machine learning. It is not difficult to write C code that is faster than standard R or Python libraries by optimizing array allocations and removing checks which are known to be satisfied in 5e context of a particular application. Doing so can slash the running time even of C++ STL functions by a factor of 10 with minimal effort. But what would a reg monkey know.

Bollocks. Really fast code would need to be massively parallelized, and optimized to run on CUDA. Google and the rest of FANG has a thousand CS PhDs working on this stuff, optimizing libraries that are actually used in production. You're delusional if you think your amateur C code can beat that.

4 months ago # QUOTE 3 Volod 0 Vlad !
Economist
232d

Why would you not?

At a certain point ggplot is ms paint- I’m feeding it a list of values hoping it renders. At that point, excel is better

or t(matrix(1:9, 3, 3)). Also what kind of projects are you working on that inputting a small matrix is the bottleneck? I don't think anything matches dplyr and ggplot2 in ease of use for data manipulation and plotting. >

If you want your plots to look like Excel plots, Excel is better

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
699f

Also the structural C++ bro is missing the point of the thread. If you have a niche use and you have the skills go nuts on C/C++. For the average reg monkey R is actually much better and produces nicer plots than Stata.

The real reason to use Stata is that economists accept all its defaults as correct (except the need to ,robust).

4 months ago # QUOTE 1 Volod 0 Vlad !
Economist
39fe

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

Would it really take that long? An econ model?

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
4325

For elegant statistical modeling, R is the absolute #1.

It is perfect for data management, model estimation and numerous follow-up analysis (incl. graphs).

4 months ago # QUOTE 4 Volod 0 Vlad !
Economist
6eb3

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

Would it really take that long? An econ model?

It is a purely made up example. Any particle filter *available* in R is *implemented* in C. For example: https://github.com/kingaa/pomp/tree/master/src

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
469b

There is no equivalent to shiny or rstudio server in python.

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
232d

Portfolio construction certainly takes that long.

Here is a simple example: implement a particle filter in R (or Python). Depending on the data set and model complexity (including the cost of evaluating the model), a single run of the particle filter may take several seconds. Now imagine you need to run the filter one million times, e.g. to estimate the model. In such cases a compiled code may slash the running time by a factor of 100, and the resulting procedure may still take several days to run. That would be several months in R or Python.

Would it really take that long? An econ model?

4 months ago # QUOTE 0 Volod 0 Vlad !
Economist
232d

There is no equivalent to shiny or rstudio server in python.

Jupyter

4 months ago # QUOTE 0 Volod 0 Vlad !

« Previous 123 4 Next »

Recently Announced

Upcoming

Economics

Job Market

Off Topic

Job Market Wiki

Job Market Candidates

Updates

Why doesn't R ever get any love?

Reply »