You can but you’d have to upload the zip file of the plugin’s old version, rather than search for the plugin and click install. Beware that uninstalling a plugin *may* remove it’s settings, so you might need to setup you the stripe plugin again.
If you have cpanel or ftp access, you can just overwrite the plugin’s files with the 3.0.2 versions and all settings will definitely remain intact.
Using v3.1.3 and importing products with 32 columns of data I can confirm that I’m also importing at roughly the exact same speed as Simonmm.
I might try downgrading to v3.0.5, would this cause any problems with the import taxonomies and custom fields add-ons (which I am also using)?