Is there a way to turn off the ad popup that appears when you close a Steam game?

It's actually quite easy to disable them. In the Settings window, go to the Interface tab. There's an option at the bottom to disable these notifications.

Notify me about new and upcoming releases.

I do believe that this is an option they added in semi-recently, because I've been using Steam for a long time and I've never seen it before a few days ago.