iMovie 9.0: how do you make clips shorter than 5ms?

I don't think it's possible to make the clips shorter then 5ms in any video software, because depending on the frame rate a single frame takes more time (16,6 ms for 60 fps). It doesn't make much sense to add a clip to a video, that is so short that it is not seen on screen for at least one frame.

I don't have an official document anywhere outlining this as a specific limitation, but having used iMovie for a while, and having just tried to replicate what you're trying to do, I'm fairly confident in saying that this isn't supported in iMovie. Final Cut will certainly support this sort of fine-grain clip control, but iMovie is very consumer orientated and I wouldn't expect it to support something as niche as this. Perhaps it's worth giving Final Cut Express/Pro a try to see if it meets your requirements?