Whitelist Issue

I have been running into an issue lately where certain domains / IPs continue to get flagged as Spam, even though they have been whitelisted through the Clean Talk plugin.

It seems to be resolved for a week or two and then the person will get flagged as spam again. Do you have any advice on how I could prevent this?