What does "the rates may be adjusted by the greater of 2%" mean in a contractual context?

It means that it was written by someone who was not a native speaker of English. It is not clear at all. (I would ask for the contract in the original language and have it translated.)

I would guess that it means "may be adjusted by our company on each anniversary of the start date by more than 2%." - This would account for the 4% hike.

Whether the company is trying to cheat is debatable but, if my interpretation is correct, the company could increase the rate by any percentage higher than 2% and that would be dangerous if you are paying the interest, but rewarding if you were receiving it.