Meaning of "eyes rolled back"

This would be more definite with a description of what the monster actually was.

It probably means exactly what it says: the monster's eyes were rolled back, that is, upwards so far that the irises could no longer be seen.

Rolled-back eyes

Image from LookLikeAZombie.com.