Infinite Scroll has a check for is_singular(). It won’t load the front end javascript if this condition is met. It would be nice to have a filter to override this behaviour. The function to look for to manually change this is shouldLoadJavascript(). It is located in infinite-scroll.php. Won’t recommend editing plugin files directly though.