Loops in single.php & pagination

I have a problem with pagination when using a loop in a single.php file. The website has the same layout as an inbox as so I want a list of the posts displayed at the top with the “reading pane” displaying the single post upon click. I have this working without issue with the following code in my single.php…

//Display emails (title only)
<?php $paged = ( get_query_var( 'paged' ) ) ? get_query_var( 'paged' ) : 1;
$args = array(
	'posts_per_page' => 20,
	'paged'=>$paged	);
$loop = new WP_Query( $args );
if ( $loop->have_posts() ) :
/* Start the Loop */
while ( $loop->have_posts()) : $loop->the_post();
get_template_part( 'content', get_post_format() );
endwhile; 

//Pagination
posts_nav_link('∞','Newer emails','Older emails');

endif;

//Display single post
while ( have_posts() ) : the_post();
get_template_part( 'content-page', get_post_format() );
endwhile;

Now this works with the list of emails above and the single email showing below… but I cannot get the pagination to work for the life of me. I think this is because for a single post it displays the content using wp_query and this is also used for the pagination regardless of my loop above. And so because it’s a single post wp_query sees only one element and thinks there is no need for pagination (I may be wrong — this is just a guess).

I’ve tried hacking it like so:

$temp = $wp_query; $wp_query = $loop;

And then at the end of the main loop (before the single post is displayed)…

$loop = null; $wp_query = null; $wp_query = $temp;

But this just results in a fatal error…

Would really appreciate some help as to how to get round this! Is there a way I can tell pagination to use a specific loop (i.e. not wp_query)…?

Thanks!