Actually this may be an issue with my 4.0 upgrade. jQuery is not defined anywhere in the admin panel and other stuff is broken as well. Reverted to 3.x and it works.
I found that you get a 404 on a particular email hash if that email address is not a primary email address. If you switch that email to be primary, it then finds it (no 404). This is actually the intended behavior according to the Gravatar FAQ. IMO, that restriction is ridiculous.