Teaching evaluations are a scam « Economics Job Market Rumors

3 years - 108 posts - Latest - RSS
Thread: 322 Goods vs 111 No Goods

Economics Job Market Rumors » Economics » Teaching

Teaching evaluations are a scam

3f50
Rep: 55

As above, an interesting article on evaluations in the JPE: "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors,"
https://www.jstor.org/stable/10.1086/653808 . Here's a key part of the introduction: "Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses)."

more beeching by crummy teachers.
try a bit harder ok?

1 year ago # QUOTE 1 Jab 1 No Jab !
Economist
24ac

That study has been replicated in different countries and subjects. The only quality evidence we have is that student evaluations lead to less learning.

My guess is that evaluations are helpful for catching people on the far left tail. But otherwise...

In my own experience, I've had my highest ratings when I've done the crappiest job teaching, but made the course easy and "fun". I've had my lowest ratings when I thought I was doing the best job teaching, but the course was challenging. Any guess how I teach now?

As above, an interesting article on evaluations in the JPE: "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors,"
https://www.jstor.org/stable/10.1086/653808 . Here's a key part of the introduction: "Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses)."

more beeching by crummy teachers.
try a bit harder ok?

1 year ago # QUOTE 5 Jab 1 No Jab !
Economist
071f

Some evaluation forms in some places have a question along the lines of "degree of challenge." Even Rate My Professor has something like that, although its numerical scheme has higher scores for easier grading. Anyway, those looking at the evals should look at that score, if there is one. Or one can simply look at the grades being given.

Here is one way to do this. Where I teach faculty competing for special awards are asked to provide both their numerical eval scores (1-5 range) as well as the GPAs for each class (0-4 range). Those looking at them then look eval-GPA, and this seems to correlate highly with reputation of faculty as teachrs. One should bet higher than 1 on this, which says the students are rating the favulty as better than the faculty is rating the students. Getting a above a 1.5 on this is very good, and getting above a 2 is really outstanding.

1 year ago # QUOTE 0 Jab 0 No Jab !
3f50
Rep: 55

There seems to be widespread dissatisfaction with how student evaluations are conducted and some universities are looking at alternatives. One summary is "Colleges Are Getting Smarter About Student Evaluations. Here’s How." https://www.chronicle.com/article/Colleges-Are-Getting-Smarter/245457 (perhaps gated). The focus of the article is a new evaluation system at the University of Oregon, where interest was spurred by an economist. Details on the Oregon's revision can be found at https://provost.uoregon.edu/revising-uos-teaching-evaluations .

Briefly, students are given the following list and are asked to both describe which one most helped them learn and which one most needs to be improved in the course: Inclusiveness, Transparency of instructions & grading, Timing of feedback, Challenge of the course, Quality and accessibility of course materials, Support, Engagement in assignments or projects, Active Learning, Quality of the interactions between students, Instructor communication, Other. Evaluations are given at both the end of the semester and in the middle so that faculty have time during the semester to make adjustments for current students. The idea is to give the instructor feedback on how to improve a course. A key part of this system is that numerical scores are not generated.

Overall evaluations at Oregon also include self-evaluations and peer review.

1 year ago # QUOTE 1 Jab 0 No Jab !
Economist
7254

I think it is only relevant when the score is way below the bar. Otherwise, who cares..

1 year ago # QUOTE 0 Jab 0 No Jab !
Economist
eda0

I think it is only relevant when the score is way below the bar. Otherwise, who cares..

When I've gotten bad e-vals, it was because I assigned "too much" work. Here's the thing: this caused my students to do more work, and thus learn more. When I assign less, my e-vals go up. It's a mechanical relation. Assigning less work is also less work for me, so there's a Nash eq. cooperative corner solution: assign very little.

1 year ago # QUOTE 0 Jab 0 No Jab !
Economist
23f6

No offense, but this is way too complicated.

There seems to be widespread dissatisfaction with how student evaluations are conducted and some universities are looking at alternatives. One summary is "Colleges Are Getting Smarter About Student Evaluations. Here’s How." https://www.chronicle.com/article/Colleges-Are-Getting-Smarter/245457 (perhaps gated). The focus of the article is a new evaluation system at the University of Oregon, where interest was spurred by an economist. Details on the Oregon's revision can be found at https://provost.uoregon.edu/revising-uos-teaching-evaluations .
Briefly, students are given the following list and are asked to both describe which one most helped them learn and which one most needs to be improved in the course: Inclusiveness, Transparency of instructions & grading, Timing of feedback, Challenge of the course, Quality and accessibility of course materials, Support, Engagement in assignments or projects, Active Learning, Quality of the interactions between students, Instructor communication, Other. Evaluations are given at both the end of the semester and in the middle so that faculty have time during the semester to make adjustments for current students. The idea is to give the instructor feedback on how to improve a course. A key part of this system is that numerical scores are not generated.
Overall evaluations at Oregon also include self-evaluations and peer review.

1 year ago # QUOTE 0 Jab 1 No Jab !
Economist
23f6

I had my highest rating when I replaced a really bad teacher during the semester.

That study has been replicated in different countries and subjects. The only quality evidence we have is that student evaluations lead to less learning.
My guess is that evaluations are helpful for catching people on the far left tail. But otherwise...
In my own experience, I've had my highest ratings when I've done the crappiest job teaching, but made the course easy and "fun". I've had my lowest ratings when I thought I was doing the best job teaching, but the course was challenging. Any guess how I teach now?

As above, an interesting article on evaluations in the JPE: "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors,"
https://www.jstor.org/stable/10.1086/653808 . Here's a key part of the introduction: "Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses)."

more beeching by crummy teachers.
try a bit harder ok?

1 year ago # QUOTE 0 Jab 0 No Jab !
Economist
93f0

Everyone freaks out about teaching evals, but education is a market. Unless you’re at an Ivy or flagship state school, your department’s future is not guaranteed. It’s financial viability directly relies on student enrollment, and therefore, student satisfaction. Teaching quality plays a role in that. Until you bring in a majority of your salary through grants and contracts, you can’t really complain.

1 year ago # QUOTE 1 Jab 1 No Jab !
Economist
f95a

this is why hard, standardised exams are a fantastic idea. if the assessment is independent and rigorous, teaching quality actually matters. otherwise it's meaningless in the cowboy world of self regulation (the UK).

academics HATE the idea of course, even though it's entirely feasible to standardise the undergraduate economics curriculum and most other subjects.

1 year ago # QUOTE 1 Jab 0 No Jab !
Economist
23f6

standardised exams are for exam taking robots who love multiple choice exams.

this is why hard, standardised exams are a fantastic idea. if the assessment is independent and rigorous, teaching quality actually matters. otherwise it's meaningless in the cowboy world of self regulation (the UK).
academics HATE the idea of course, even though it's entirely feasible to standardise the undergraduate economics curriculum and most other subjects.

1 year ago # QUOTE 1 Jab 2 No Jab !
Economist
23f6

You're a parasitic administrator. Please die.

Everyone freaks out about teaching evals, but education is a market. Unless you’re at an Ivy or flagship state school, your department’s future is not guaranteed. It’s financial viability directly relies on student enrollment, and therefore, student satisfaction. Teaching quality plays a role in that. Until you bring in a majority of your salary through grants and contracts, you can’t really complain.

1 year ago # QUOTE 0 Jab 2 No Jab !
Economist
f95a

standardised exams are for exam taking robots who love multiple choice exams.

this is why hard, standardised exams are a fantastic idea. if the assessment is independent and rigorous, teaching quality actually matters. otherwise it's meaningless in the cowboy world of self regulation (the UK).
academics HATE the idea of course, even though it's entirely feasible to standardise the undergraduate economics curriculum and most other subjects.

who said multiple choice?

they are from perfect but they are the best option, something an economist should be able to grasp.

1 year ago # QUOTE 0 Jab 0 No Jab !
3f50
Rep: 55

No offense, but this is way too complicated.

Most now get a report like this: 4.5/5 for quality of course and 4.7/5 for quality of the instructor. What can you do with this?

At Oregon, your midterm evaluation might say that 70% of your students think your use of active learning is helping them learn, 20% feel that that quality of instructional materials is helping them learn, and 10% report a variety of other things. On what can be improved, 50% report instructor communication, 40% report timing of feedback, and 10% report other items. The final course evaluation would have a similar report.

I don't really see what's complicated and it certainly would do more to help one teach better than the current system.

1 year ago # QUOTE 1 Jab 0 No Jab !
Economist
6a0d

This.

I’ve been teaching 25 years. When I parrot the textbook and broadcast the exam in advance, I get 4.5. Whenever I deviate from this - especially if it involves requiring that the students engage in *independent thinking* things nose-dive.

That study has been replicated in different countries and subjects. The only quality evidence we have is that student evaluations lead to less learning.
My guess is that evaluations are helpful for catching people on the far left tail. But otherwise...
In my own experience, I've had my highest ratings when I've done the crappiest job teaching, but made the course easy and "fun". I've had my lowest ratings when I thought I was doing the best job teaching, but the course was challenging. Any guess how I teach now?

As above, an interesting article on evaluations in the JPE: "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors,"
https://www.jstor.org/stable/10.1086/653808 . Here's a key part of the introduction: "Student evaluations are positively correlated with contemporaneous professor value-added and negatively correlated with follow-on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value-added in follow-on courses)."

more beeching by crummy teachers.
try a bit harder ok?

1 year ago # QUOTE 1 Jab 1 No Jab !
Economist
93f0

You're a parasitic administrator. Please die.

Will you still be championing the free market when your trash department is closed and your tenure is rescinded due to low enrollment?

1 year ago # QUOTE 1 Jab 1 No Jab !
Economist
a14b

I would argue that a focus on keeping students happy is myopic. At the end of the day, there needs to be value from the degree. If we make courses too easy then they don't learn anything and are less competent employees. The degree also loses signaling value! This makes students' after-graduation salaries lower (relative to more difficult courses) and drives many of them into other fields. I hear all the time from my industry friends that our students are less and less prepared; and, their starting salaries show it.

So...yes, I will still champion the free market.

You're a parasitic administrator. Please die.

Will you still be championing the free market when your trash department is closed and your tenure is rescinded due to low enrollment?

1 year ago # QUOTE 2 Jab 2 No Jab !
Economist
c9e2

In my department, the two headline questions are overall quality and level of challenge. What we are interested in is something fairly close to the minimum of the two.

Some of our faculty get 4.5+/5 on the overall quality but less than 3/5 on the level of challenge. They are not ranked as highly on our teaching metrics as those of us who get low 4s on both.

I think it is a decent system. We want faculty to challenge students without making too many of them miserable.

1 year ago # QUOTE 0 Jab 0 No Jab !
3f50
Rep: 55

I think it is only relevant when the score is way below the bar. Otherwise, who cares..

A lot of people. Mid to high scores are commonly used for P&T and raises outside HRMs. Plus, many economists have teaching focused jobs and using a poor evaluation system doesn't benefit a soul.

There is also the off chance that a better evaluation system might lead to better teaching and thus more learning by our students...

1 year ago # QUOTE 1 Jab 0 No Jab !
Economist
23f6

A good chunk of students don't care about learning.

I think it is only relevant when the score is way below the bar. Otherwise, who cares..

A lot of people. Mid to high scores are commonly used for P&T and raises outside HRMs. Plus, many economists have teaching focused jobs and using a poor evaluation system doesn't benefit a soul.
There is also the off chance that a better evaluation system might lead to better teaching and thus more learning by our students...

1 year ago # QUOTE 0 Jab 0 No Jab !

« Previous 1 234…6 Next »

Recently Announced

Upcoming

Economics

Job Market

Off Topic

Job Market Wiki

Job Market Candidates

Updates

Teaching evaluations are a scam

Reply »