Is there an antonym for "delegate"?
Solution 1:
I think principal might work for you. It's used fairly frequently to describe the primary people involved in things and the following is listed as one of the (chiefly legal) definitions:
a person who authorizes another, as an agent, to represent him or her.
Solution 2:
In the context of elections, those who send a delegate are generally referred to as their constituents. That may be appropriate for this context.