We managed to resolve this:
We were inserting our comment replies with:
wp_insert_comment()
The plugin hooks into
wp_new_comment()
So replacing those made it work straight away.
Thanks Ronald, I’ve left you info on WordPress Slack DM as it requires login credentials.