Why is everybody suddenly scared of me at Lady Boyle's Last Party?
If anyone sees you while you're trespassing (upstairs) everyone will turn hostile (one person seeing you = everyone, oddly enough), I'd bet that's what's happening. It can be tricky to come back downstairs without being spotted (even when using tricks like timestop and blink people seem to see me). Note that you can totally kill/knock out the upstairs guards without alerting everyone else if you're careful; every playthrough I've done I've KOed or killed the guards upstairs and I usually get out scott-free.
Try exiting the downstairs via a more discrete way: you can leave via the balcony (the door to the outside from the first bedroom after the stairs) and the guard outside won't "detect" you (he's also easy to knock out or kill, but it's pretty unnecessary). He just politely asks you to stop trespassing. Drop gracefully from the balcony and no one should turn hostile (note that falling on someone's head, as I did the first time, counts as an attack).
Also note the basement, oddly, doesn't count as trespassing, so I'd poke around all the "safe" areas before going upstairs, and making a real save before going upstairs, since it's usually when things go south in this mission.
It appers that it's just a bug, triggered by who knows what. Boyle sisters mission seems to be preatty messy, lots of bugs have been found by players. I had that problem too and I was never seen nor guards where found unconscious. Stealing shouldn't be a problem since I've stolen tons of stuff before and never had problem with guards going randomly aggro. Besides it would be contrary to the philosofy of the game. If they didn't see you, you didn't do it. So, sadly, just a bug. I'm going to sart the mission again nad trying to avoid going upstairs...
Stealing is also a problem. If you steal too much some staff may become hostile to you and that will cause problems. Also some guests will become aggressive if you are around them too much and they recognize you.