Why won't Preston Garvey follow me back to the castle?

I started the castle mission for the Minutemen too early and was not able to complete the mission. After I fought the mirelurks and destroyed every last one of the eggs, I left mid-mission to do other quests. I traveled to Mass Pike Interchange with Preston and another companion and now Preston is stuck at Mass Pike Interchange

Now, when I try to go back, it tells me I must destroy all the egg clusters. Can anyone help me with this? I've already tried leaving the area and waiting a few days, but he's still stuck.

You said you destroyed all of the eggs, is it possible you only destroyed the ones in the courtyard and missed the ones on the inside of the castle? If this is the case perhaps you have not completed the quest and the game is not working properly. Little tip, I would advance well armed and with power armor.