Searching for emoji and symbols

Uh, I always figure these things out myself immediately after posing the question. Here's the solution - I hope it will be helpful for someone else:

To search for characters, you just press ctrl-CMD-space to bring the dialog up and then start typing. The search bar appears as soon as you type something, along with the button that switches to the "character viewer" window. They will then be there the next time you bring the dialog up as well.

This is actually more convenient than the old dialog, it's just that for some reason they went out of their way to make the feature hard to discover.