How do I send invitations for Calendar events with locations?

This turns out to be a known bug. Apple Support's recommendation is (astonishingly) simply:

Do not add a location to events you plan to invite people to.

Apple is "working on it as we speak".

I just discovered I have this same problem. But in my case, while it started with not being able to send invites when there was an address, now it will not send invites at all regardless of whether there is an address. A have Apple support working on this at the engineer level. I am running Yosemite and found the problem occurs whether I create an event in the Mac OS Calendar App or log in to iCloud and create event there. The senior tech advisor had be go though these steps: https://support.apple.com/en-us/HT204598

I offer this in case it might help your situation (it did not help mine).