My pagination doesn't work with query_posts()

I have 2 “blog” pages on my new site. The home page itself and a page titled blog with the following code:

<?php
	// Query posts
	query_posts("posts_per_page=5");
?>
<?php if ( have_posts() ) : while ( have_posts() ) : the_post(); ?>
	<!-- all the loop stuff -->
<?php endwhile; endif; ?>

So what I do to overwrite the default page behavior, as you can see, is call query_posts() without including the $query_string variable.

However this screws up my pagination some how. When I call the second page it simply fetches the same posts over and over again. You can try this by going to http://ahrengot.com/blog/ and then navigating to http://ahrengot.com/blog/page/2 or by clicking the “show more posts” button which does the same thing, but loads the page via ajax (Using Paul Irish’ Infinite Scroll plugin to do this).

I figure that I’m going at this from a wrong angle, so I would love any pointers in the right direction!

Many thanks in advance 🙂