Haven't you been paying attention. There is one specific allegation of him groping (a very specific act) a PhD student, GM, at a known workshop in San Diego in 2015, where GM told many others when it happened. So there are people that can corroborate the date and location. Can't be more clear than that. I'm sure that the ESA and AEA have more examples but obviously they are not going to announce them here.
What the exact punishment is depends on the strength of evidence on what exactly counts as assault. I think the proper role of ESA and AEA should be to establish guidelines for punishments for specific offenses and degrees of evidence...especially because of how polarized the conversation is.
GOssip about allegedly bad behavior is fine and normal. It's actually a good check against creepy behaviour, so long as punishments are not dolled out on that basis. In many cases "dont work with that creep" is closer to the right degree of punishment to "terminate and excommunicate".