Worked like a charm

Tried this and w3tc. w3tc gave me a much higher pagespeed score, but the site itself became incredibly slow. SO even though the numbers said it was all great, the plugin basically sucked. I paid someone to get all the optimum settings etc but it was still slow in a real environment.

Super Cache might not give the best pagespeed rank but my site became really fast.
Combined with pagespeed server I get load times of under 1 second.
I get around 2K unique visitors per day.