Never mind. I realized that it was mod_pagespeed which was inserting the header. Please ignore this issue.
fixed it!
sudo vi /etc/apache2/sites-available/default
change
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
to
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride ALL
Order allow,deny
allow from all
</Directory>
Restart apache
sudo /etc/init.d/apache2 restart
Also check your .htaccess file, for some reason my generated .htaccess had the wrong RewriteBase
My folder is called wordpress under /var/www, so I changed the RewriteBase in .htaccess to
RewriteBase /wordpress/