"In case" vs "if"?
I don't think it's inherently incorrect. Depending on the context you could use in case in a sentence in a past tense (e.g. Please check your phone's pictures in case you have seen this man but didn't notice).
I think it's just the sentence you chose makes it hard to find a fitting context.
With this I don't mean to imply that if and in case are interchangeable, just that you could use in case in that sentence without it being incorrect.