There is an option to defer javascript globally, or for pagespeed only.
Async means that scripts will load out of order, and in most cases it will break things because some will load before jquery, inevitably.
Scripts on wordpress need to follow a specific order of dependencies.
You can only async, js files that can load completely independent from others, without dependencies… hence FVM, only uses defer to keep the order of execution.
Also note… most of the time jquery needs to be render blocking, or your page may nor work properly, if there is any inlined jquery on the theme or plugins.
I suggest reading:
https://stackoverflow.com/questions/10808109/script-tag-async-defer
