Just a note, this is actually a failure on the part of the developers who created this. I have been working on optimizing the database servers at the hosting company I work at and keep running into this (I found your thread here while trying to track down whoever is responsible for the coding). You don’t really need to cut down on the data if the queries are properly indexed – just do this from the mysql commandline and it should fix it (run an explain [query] before and after and you’ll see the difference):
alter table wp_bad_behavior_log add index ip_http_response (ip(10),http_response);
That should fix it, I just fixed one that went from examining 200,000 rows 1 (that one included the date so the index had to be (ip(10),http_response,date).
