launchctl starts my plist job much later than StartCalendarInterval

Since your per-user launch agent does actually execute, albeit later than scheduled, it is highly likely that your system may be asleep at the scheduled time. I would recommend reading through the man page by running the following command from a terminal session:

$ man 5 launchd.plist

Regarding the StartInterval option specifically, the man page has this to say:

Unlike cron which skips job invocations when the computer is asleep, launchd will start the job the next time the computer wakes up. If multiple intervals transpire before the computer is woken, those events will be coalesced into one event upon wake from sleep.

If you are working remotely, or not physically present at the scheduled time — which is not clear from your question — a good starting point would be to confirm that the system has not entered sleep mode prior to the scheduled time of your launch agent.

Note that the documentation for StartInterval has changed as of 10.10, see https://forums.developer.apple.com/thread/23361 for a thread documenting this change.

The new text reads

StartInterval

This optional key causes the job to be started every N seconds. If the system is asleep during the time of the next scheduled interval firing, that interval will be missed due to shortcomings in kqueue(3). If the job is running during an interval firing, that interval firing will likewise be missed.

Apparently this has always been 'broken' and the documentation has been updated to reflect that.

So the answer is that StartInterval doesn't handle sleep, and you can either prevent sleep (as you found) or switch to another method to trigger your job. For instance StartCalendarInterval does coalesce events after returning from sleep, so that might be a good option.