How to interpret the meaning of "I am surprised that..."?

When somebody says "I am surprised that...", does it mean to express not only surprise but also some level of disappointment or disapproval?

If somebody uses that expression when speaking with me, I feel somewhat annoyed and guilty. It feels like there is some subtle expression of disapproval and even annoyance on the part of the speaker that they can't openly express.

Would a native speaker of English interpret this expression the way I do? Or would it depend not only on the expression but also on the manner of expressing it?

In principle it might be expressing either approval or disapproval, depending on context; but contra Jasper Loy I think it is more likely to express disapproval, because if it expresses approval I think it would be likely to be accompanied by words making that approval clearer.

So

"Wow! I'm surprised you said that"

is probably impressed, but

"I'm surprised you said that."

is probably disapproving, though either of these inferences may be changed by a suitable tone of voice.

I am surprised that XXX simply means XXX evokes surprise. This surprise can be a good or bad one. The expression itself does not tell you which it is, and you can choose to infer from the context.

For example, suppose there is a girl you like and you somehow managed to find out her email without asking her for it. She could be pleasantly or unpleasantly surprised to receive your email. In either case she could still say

I am surprised that you found out my email.