Pagination with multiple loops dont work

Hi
This is what Im trying to accomplish:

Display four posts from one category and then one post from another category and so on:

cat 1
cat 1
cat 1
cat 1
cat 2
cat 1
cat 1
cat 1
cat 1
cat 2
…
I use two loops based on query posts inside the standard loop. It all works but my pagination fails. If I set post_per_page to for example ten, WP displays 10 posts of posts from both categories (ten posts from cat 1 ten posts from cat 2). I have limited the second category to only show post_per_page/4 but thats only a bad solution. I should also mention that I have a lot of categories and sub-categories that I dont display and in the beginning of the code I loop trough these and put them in arrays so the query know which posts to loop trough. Here is the code, any solutions??

[Code moderated as per the Forum Rules. Please use the pastebin]