Disable WordPress Login for certain users/roles

You should be able to hook into the authenticate filter at the very end (after all other hooks have run, including Authorizer and core WordPress authentication), and then check whether you have a successful authentication of a user with administrator role but not a value of google in the authenticated_by user meta (Authorizer adds this user meta for external service users).
https://developer.ww.wp.xz.cn/reference/hooks/authenticate/

So something like this (untested, just wrote this now):

add_filter( 'authenticate', function ( $user, $username, $password ) {
	// Fail to authenticate administrator users unless via Google in Authorizer.
	if ( 
		! empty( $user->roles ) &&
		in_array( 'administrator', $user->roles, true ) &&
		'google' !== get_user_meta( $user->ID, 'authenticated_by', true )
	) {
		$user = new WP_Error( 'authentication_failed', __( 'Admins must authenticate with Google.' ) );
	}

	return $user;
} ), PHP_INT_MAX, 3 );

Just be careful with this filter since it’s easy to accidentally lock yourself out of WordPress.