I don’t know if this is relevant to this discussion, but I’m having a similar problem. I have a theory. The OP says that his site is http://www.tammasz.com.
I have several wordpress sites on the same web host.
I have only been able to get activitypub to work on those sites that are on domains without subdomains. So my theory is that while https://www.tommasz.com doesn’t work, it will work on https://tommasz.com.
Of my 5 sites, the three without subdomains (domain.com) work fine out of the box. The other 2 (www.domain.com) do not work.
I suppose there could be 100 reasons, but thought I’d offer this to see if it might ring a bell somewhere.