What is Khadgar saying?
What exactly can I not return to?
Azeroth. What he's talking about is that after you cross through the Dark Portal, you have to play through a (roughly hour-long) introductory quest chain before you can leave. Shortly after you complete the intro sequence, you should acquire the means to return to Azeroth.