WP_Query with specific posts_per_page value returns more than specified

For some reason, whenever I construct a new WP_Query with arguments set for post_type, post_status, posts_per_page, and paged I don’t get the correct number of posts back.

Take my code here for example:

$blogPostFeed = new WP_Query([
   'post_type' => 'post',
   'post_status' => 'publish',
   'posts_per_page' => 2,
   'paged' => 1,
   'orderBy' => 'date',
   'order' => 'DESC'
]);

After running this query, I get 4 posts that come through the loop. If I change the posts_per_page value to 3, I get 5 posts that come through the loop.

What would cause this kind of behavior?