Bestiary catching system
1) If the hp of the monster is not low enough 3 seconds after throwing the net it will rage and escape, this threshhold depends on the type of net en monster. I don't know the threshhold. But making it run out of hp while netted will guarantee a catch.
2) I think there is a visual cue, but I'm not sure, I'll check tonight and update the answer.
3) As far as I know the only effect of raging is that it is impossible to catch for a few seconds. And it heals up some part of it's hp.