Another flawed JF paper? « Economics Job Market Rumors

10 years - 120 posts - Latest - RSS
Thread: 5 Goods vs 7 No Goods

Economics Job Market Rumors » Job Market » Finance Job Rumors

Another flawed JF paper?

Economist
6096

hmmm...and how about f**k off, old fart?

- the authors

The study by Lacerda and me (Eriksson) did not contain any testing of the commercially available product. What we did do, was emulate the technology in Mathematica to get a better feeling for its workings. The emulation was based on the very detailed description in the patent documents; a description the producers of the technology (Nemesysco) have repeatedly confirmed is indeed the core engine of the commercial product. When they say this is just 5% of the code, this is really a meaningless piece of information. What is not in the code are things like the graphical interface and the like which is completely irrelevant with respect to the validity and reliability of the technology.
Our emulated version was developed only to get a better idea of typical outputs given varied input data. It was not a reliability test. There are two reasons why we did not do that. There are some very sophisticated studies out there (see references in Eriksson and Lacerda 2007) testing the reliability of the product, and they come to the conclusion that it performs at chance level. Secondly, if the methodology is totally invalid it makes no sense wasting time on reliability testing.
You checked out an emulated version in Mathematica? Who did the emulation and who checked the code? And how the hell do you even know what to look for? Let me be clear about why I find your Mathematica analysis irrelevant: I did my undergrad in electrical engineering and spent some time in our digital audio lab. Even with what I know from that, I wouldn't feel confident judging this software -- and I'm a hell of a lot more competent than you in that area.
You guys were a bit too aggressive with a flawed technique. You screwed up. Hey, it happens; but, the true measure of you both is how you handle that. So email the Swede, apologize for being bristly, and work *with* him to use a better method. Hell, you can probably even get him to help you make the switch. But do not come here and try to justify your actions and work which seem deeply flawed. That's the weak, easy, and pathetic path.
This is one of those times in life where you are given the difficult choice of publicly admitting a problem and fixing it -- or fooling only yourselves by saying nothing is wrong. Grasp the nettle, revise the work to put it on a solid foundation, and I suspect you will impress everyone who ever reads about what happened.

10 years ago # QUOTE 0 Volod 9 Vlad !
Economist
07b1

Which authors? If you are the JF authors, then go f**k yourselves.

10 years ago # QUOTE 2 Volod 0 Vlad !
Economist
33a2

There is some confusion in this discussion on the Mathematica emulation. Lacerda has actually a degree in electrical engineering, telecommunications and electronics, and also a PhD in phonetics. His Mathematica emulation of the LVA patent was just to check how the LVA-variables based on the “thorns”, “plateaus” and the poor quantization of the speech signal would come out in relation to actual phonetic and signal processing measures of the speech signal. The emulation alone does not demonstrate anything essential about the validity of the LVA. Lacerda’s argument on the LVA’s lack of validity is that the thorns, plateaus and the descriptive statistics that follow, simply do not provide any relevant information. This is a fact based on signal processing principles and it was only verified by the emulation. Lacerda explains this in his PERILUS 2012 manuscript (http://www.ling.su.se/perilus/perilus-2012).
The Mathematica emulation was just a way to verify that the results from LVA were indeed meaningless when compared to relevant phonetic measures.
Hobson, Mayew and Venkatachalam’s idea is interesting but they messed it up by using that LVA black box instead of meaningful speech measures. Phoneticians know how to get relevant prosodic variables from the speech signal but finance people most probably do not and they wasted their time using the LVA’s black-box. It does not make sense to discuss their current results because there are based on a flawed technology. It does not even make sense to compare them to future results using proper speech analysis. The only sensible thing to do is to forget about the naïve mistake of believing in the LVA stuff and start over again using a proper prosodic analysis of the conference calls based on valid prosodic measures. If those valid measures can be meaningfully related to emotional states, all the better. But this is not easy and do not continue being naïve and think that you will be easy to infer emotional states from prosodic variables. Above all, learn to be suspicious about LVA and other magic devices from wonderland.

10 years ago # QUOTE 6 Volod 0 Vlad !
Economist
33a2

Correct URLs

Lacerda's PERILUS 2012 manuscript:
http://www.ling.su.se/perilus/perilus-2012

The lead article in February's JF:
http://www.afajof.org/journal/abstract.asp?ref=0022-1082&vid=67&iid=1&aid=1&s=-9999

A prof at Stockholm University says it is garbage:
http://stockholmuniversityphonetics.com/2012/03/15/lva-voice-analysis-technology-a-threat-to-the-credibility-of-academic-financial-studies/

The authors' reply is in the comments and the reply to the reply is here:
http://stockholmuniversityphonetics.com/2012/04/07/money-talks-the-power-of-voice-comment-on-william-j-mayew-and-mohan-venkatachalams-reply/

10 years ago # QUOTE 2 Volod 0 Vlad !
Economist
85c9

There is some confusion in this discussion on the Mathematica emulation. Lacerda has actually a degree in electrical engineering, telecommunications and electronics, and also a PhD in phonetics. His Mathematica emulation of the LVA patent was just to check how the LVA-variables based on the �thorns�, �plateaus� and the poor quantization of the speech signal would come out in relation to actual phonetic and signal processing measures of the speech signal. The emulation alone does not demonstrate anything essential about the validity of the LVA. Lacerda�s argument on the LVA�s lack of validity is that the thorns, plateaus and the descriptive statistics that follow, simply do not provide any relevant information. This is a fact based on signal processing principles and it was only verified by the emulation. Lacerda explains this in his PERILUS 2012 manuscript (http://www.ling.su.se/perilus/perilus-2012).
The Mathematica emulation was just a way to verify that the results from LVA were indeed meaningless when compared to relevant phonetic measures.
Hobson, Mayew and Venkatachalam�s idea is interesting but they messed it up by using that LVA black box instead of meaningful speech measures. Phoneticians know how to get relevant prosodic variables from the speech signal but finance people most probably do not and they wasted their time using the LVA�s black-box. It does not make sense to discuss their current results because there are based on a flawed technology. It does not even make sense to compare them to future results using proper speech analysis. The only sensible thing to do is to forget about the na�ve mistake of believing in the LVA stuff and start over again using a proper prosodic analysis of the conference calls based on valid prosodic measures. If those valid measures can be meaningfully related to emotional states, all the better. But this is not easy and do not continue being na�ve and think that you will be easy to infer emotional states from prosodic variables. Above all, learn to be suspicious about LVA and other magic devices from wonderland.

I realize I am partly to blame for this confusion about “the authors” by not being clear enough in my posting (2eo4 above). I am the Eriksson behind the Eriksson and Lacerda, 2007, paper that caused quiet a stir some years ago when we exposed the Nemesysco technology as invalid.

10 years ago # QUOTE 5 Volod 0 Vlad !
Economist
3e28

What a shame for JF.
Editorial board should make some decisions...

10 years ago # QUOTE 9 Volod 1 Vlad !
Economist
632f

There can be no shame in publishing Duke papers in JF.

10 years ago # QUOTE 1 Volod 1 Vlad !
Economist
88fc

To Eriksson: thanks for posting. Do you know whether the Duke authors and/or the JF editors knew about your paper before the decision was made to publish the Power of Voice paper in the JF?

10 years ago # QUOTE 3 Volod 0 Vlad !
Economist
33a2

To Eriksson: thanks for posting. Do you know whether the Duke authors and/or the JF editors knew about your paper before the decision was made to publish the Power of Voice paper in the JF?

Yes, they must have known, obviously. The JF paper refers to Lacerda's and Eriksson & Lacerda's work but dismiss it in favour of Nemesysco's solution. Lacerda mentions this in his blog at http://stockholmuniversityphonetics.com/2012/03/15/lva-voice-analysis-technology-a-threat-to-the-credibility-of-academic-financial-studies/

10 years ago # QUOTE 5 Volod 0 Vlad !
Economist
2f05

What do you expect from the current JF editors?

10 years ago # QUOTE 2 Volod 0 Vlad !
Economist
7fa8

I find this thread to be interesting, but not because it lists JF papers with mistakes. This is to be expected. Do you really expect the error rate at a journal to be zero? What I find interesting is the demonization of the JF editors. I personally do not think they are doing the greatest job in the world, but even if they were, they would make mistakes from time to time. So list away. It's useful to know which papers are incorrect, but nasty comments directed at the editors are unwarranted.

10 years ago # QUOTE 3 Volod 3 Vlad !
Economist
88fc

Yes, they must have known, obviously. The JF paper refers to Lacerda's and Eriksson & Lacerda's work but dismiss it in favour of Nemesysco's solution. Lacerda mentions this in his blog at http://stockholmuniversityphonetics.com/2012/03/15/lva-voice-analysis-technology-a-threat-to-the-credibility-of-academic-financial-studies/

Yep, you're right. I see the Lacerda references in the paper. What was the JF thinking???

10 years ago # QUOTE 4 Volod 0 Vlad !
Economist
07b1

To Eriksson: thanks for posting. Do you know whether the Duke authors and/or the JF editors knew about your paper before the decision was made to publish the Power of Voice paper in the JF?

Agreed. And sorry for thinking you were the JF authors. It takes guts to come here and post publicly with your name. Bravo.

Since the paper has yet to be published, the editors should pull it and ask for a revision with new software. Proceeding as though everything is OK would be cowardly and a very negative signal about JF's commitment to quality.

10 years ago # QUOTE 10 Volod 0 Vlad !
Economist
33a2

Yes, they must have known, obviously. The JF paper refers to Lacerda's and Eriksson & Lacerda's work but dismiss it in favour of Nemesysco's solution. Lacerda mentions this in his blog at http://stockholmuniversityphonetics.com/2012/03/15/lva-voice-analysis-technology-a-threat-to-the-credibility-of-academic-financial-studies/
Yep, you're right. I see the Lacerda references in the paper. What was the JF thinking???

The soft talk about the LVA-technology and their circular arguments may be difficult to see through for reviewers that do not know about speech. It's no excuse, they might have asked for an independent opinion of someone from speech analysis, but makes the mistake more understandable. Forget about the LVA stuff and start over using scientific software, like PRAAT, that you know what it is doing. That would make a meaningful contribution.

10 years ago # QUOTE 3 Volod 0 Vlad !
Economist
94c3

Because they are biased

I find this thread to be interesting, but not because it lists JF papers with mistakes. This is to be expected. Do you really expect the error rate at a journal to be zero? What I find interesting is the demonization of the JF editors. I personally do not think they are doing the greatest job in the world, but even if they were, they would make mistakes from time to time. So list away. It's useful to know which papers are incorrect, but nasty comments directed at the editors are unwarranted.

10 years ago # QUOTE 0 Volod 1 Vlad !
Economist
1142

You are not familiar with finance research, are you?

How did the Duke authors find significant results if the measure is simply noise? The Swede didn't run any tests, he simply argued that the commercial measure is of poor quality.

10 years ago # QUOTE 2 Volod 2 Vlad !
Economist
7b79

This is an excellent thread discussion...thank you for posting. It was a fascinating read.

Personally, i find it hard to believe the paper made it through the JF refereeing process. It is hard to buy the argument that you can actually tell someones real underlying emotions about something by using voice analysis software, let alone a black box which is not verifiable. I think it got through on the novelty of the approach at the expense rigorous scientific method. This is not good for the credibility of our field and makes us the laughing stock of academic research disciplines....especially since the JF is the premier journal in our field. Let this be a lesson to the incoming JF editors!

10 years ago # QUOTE 7 Volod 0 Vlad !
Economist
e89e

I thought Lacerda was a Portuguese photographer.

10 years ago # QUOTE 6 Volod 0 Vlad !
Economist
de31

I thought Lacerda was a Portuguese photographer.

Husquavarnas, Yamahas, DMZs,..

10 years ago # QUOTE 5 Volod 0 Vlad !
Economist
3e77

Since the paper has yet to be published, the editors should pull it and ask for a revision with new software. Proceeding as though everything is OK would be cowardly and a very negative signal about JF's commitment to quality.

The paper has already been published. As noted before, it was the lead article in February's JF:

http://www.afajof.org/journal/browse2.asp?ref=0022-1082&vid=67&iid=1&oc=&s=

10 years ago # QUOTE 0 Volod 0 Vlad !

« Previous 123…6 Next »

Recently Announced

Upcoming

Economics

Job Market

Off Topic

Job Market Wiki

Job Market Candidates

Updates

Another flawed JF paper?

Reply »