Is there a mod or another solution that can keep me from accidentally stealing stuff?
Yes there is, blocksteal redux will prevent you from stealing. 3,670 endorsements, too, so it should be pretty good.
To remove your bounty via console use the paycrimegold command:
player.paycrimegold
<X> <Y> <FactionID>Sometimes, does not pay off your bounty fully.
X = 1 Remove stolen items. 0 Don't remove stolen items.
Y = 1 Go to jail. 0 Don't go to jail.