Nevermind, I must have been doing something wrong. I was able to get it to work by running wp-scripts build ./src/index.js ./src/scripts.js
Nevermind, it wasn’t clear how the filtering actually works. What I should have been doing is /wp-json/posts?filter[category_name]=videos