Where are Reminders stored on disk?
While simply copying the directories mentioned in the question does not help (presumably, there is some database somewhere containing the uuids to the calendars and one has to copy that as well), it is possible to manually import the *.ics files by simply opening them.
Update
It seems that copying the folders and then rebooting actually does restore the reminders (or probably logout-login will suffice).