Duplicate Category Pages

I’ve got a problem with duplicate category pages being available. You can see the problem here:

http://www.whatmarkdid.com/category/gadgets/

http://www.whatmarkdid.com/gadgets/

Note that the top link uses the default category slug before the category name. The second link removes the category slug and WP still returns a page. As far as I am concerned the first link is correct, the second one shouldn’t return anything (it should 404). I’ve checked this across multiple sites (all using WP3.9.1) using different templates (Responsive, 2011,2012 and 2014) and they all do the same thing.

There’s two problems with this, first it’s providing duplicate content to search engines. And second, the Older Posts/New Posts links are broken on the pages that do not include the category slug.

How do I fix this?