Posts repeat

Just installed the plugin for a new site and put the short code on a test page. To test, I created a bunch of dummy posts and set the posts per page to 5 on my test page. When I scroll down new posts load, but then I have a grayed out “Load more” button and 5 posts below it. When I go back and to the test page and edit the number of posts to load, the number of posts that get displayed below the grayed out “Load more” button changes accordingly.

I’m clearly missing something – can you point me in the right direction?

Thanks!
FL