What happens if I get arrested?

When you go to jail, everything you have on you will be taken away and stored in a chest until you serve out your prison sentence.

However, you will always manage to sneak a single lockpick with you when you enter jail so you have a chance to break out. If you choose this method, you'll have to locate the chest with your equipment to reacquire it.

Should you choose to serve your sentence, just sleep in your jail cell. Keep in mind this has some consequences

The reason why the amulet was not taken is due to the fact that it was most likely not considered stolen. When you kill someone and take the stuff from their body, its not considered thievery.

Note, this is only for normal arrests/jail sentences. There is at least 1 quest that gets you sent to jail and that is treated differently.

I've found that guards do keep coming, and each time you kill them, you're just digging yourself into a deeper hole. Eventually, they won't care about the bounty anymore: they'll just attack you on sight. If you don't want to pay the bounty or go to jail, your best bet is to pull a Harrison Ford and run.

Going to jail kinda stinks, but it's not a bad way to deal with an issue. You'll lose most, if not all, of your progress in skills as well as some time. When you complete your obligation to society, you'll find yourself right outside the jail. Anything you didn't steal will be returned to you.

You also have the opportunity to create your own Fox series and stage a prison break if you want.

Jail or bounty, your stolen stuff is locked away in an evidence chest generally somewhere in the jail, but if you're especially daring, you can get them back.

If you're interested in seeing all that Skyrim has to offer, Zagat gives prisons 4 and a half stars.