sorry that I did something, sorry for doing something, or sorry to have done something?

Should it be:

1) "I am sorry (that) I did this to you."

2) "I am sorry for doing this to you."

3) "I am sorry to have done this to you."

From what I have learnt about 'sorry', I would exclude 3) 'sorry to have done this to you' because 'sorry to do' is like 'regret to do' – with the regrets coming before the deed – as opposed to 'regret doing', where the regrets come after the deed. Consequently 'sorry', like 'regret', shoud not be used with a perfect infinitive. Am I right?

Then, which to choose between 1) and 2), and why?

2) works only in the present tense, if what you are apologizing for is occurring as you speak --- for example, if you were a dentist pulling your patient's tooth, you might say "I am sorry for doing this to you", because in pulling their teeth, you are causing them pain.

Your point with regard to 3) is very technical, and comes from your mistaken notion that sorry to do is like regret to do. While they are similar, they certainly are not exact synonyms, and you do not have to use the word regret exactly as you use the word sorry. The usage of 3) is perfectly fine in colloquial conversation.

1) is also fine. You may also say "I am sorry for having done this to you."