your post even shows it was 40 seconds. I enabled caching and it loads very quickly if you’ve been to the page before. Having multiple people test on IE Safari and Firefox they all report very slow load times
I use a folder under the webroot for all non wordpress files. So the link would go to http://www.somesite.com/pages/static.html