What are the consequences of stealing? (I.E. Taking things from containers with a red hand icon)

If you steal the item successfully whilst in stealth mode, there's no punishment. If you are caught, you may just get a talking to, or you may get a reputation loss through either the action or any combat that's initiated because you were caught. This can lead to the possibility of not being able to select certain special reputation dialogue options that can appear, if you're regarded poorly with the relevant faction.