'if we were to go with them' vs 'if we were to have gone with them'

What is the difference between the two tenses? 1. If we were to go with them. 2. If we were to have gone with them.

Both are subjunctive, which means here that they're hypothetical/contrary-to-fact statements. #1 takes place in the present--the opportunity to go with them has not yet passed, and we're contemplating what might happen if we go. #2 takes place in the past, which is indicated by use of the perfect--the opportunity to go with them has already passed, and we're explaining what would have been the result of the trip that we didn't take with them.

For example:

1. They're planning a trip to Hawaii and they invited us. We're probably not going to go, because if we were to go, we would eat too much pineapple and die. We probably shouldn't go!
2. They invited us to Hawaii, but we declined. We did not go with them. If we were to have gone, we would have eaten too much pineapple and died. Thanks heavens we didn't go!