Updated. Apparently, it didn’t work if I use slugs. However if i use the full category names, it works. This would be a problem because while slugs don’t change, the category names may change since it’s front-facing.
I’m having this issue as well. I followed the example at https://wp-eventmanager.com/knowledge-base/the-event-listings/ exactly and still cannot list events by category without using filters.