Do I need to hide API key when using google maps js API? If so, how?

You can create multiple API keys with different restrictions to use them safely. For embedding a map, the Google Maps documentation has instructions for creating a correctly restricted API key so that it cannot be abused for other purposes at Get an API Key - Restricting API keys. It's OK to include a restricted API key in your source code, because you cannot embed a map properly without doing that anyway.

If you need server-side API access, you can create a second API key with less restrictions. That one should be kept secret.

While the above answers are helpful, none of them addresses the following vulnerability:

Once a user has access to your API key, even if they are restricted to using it only from your domain, they can still use it as much as they want. In the crudest sense, this could mean a million page refreshes (and map loads) in a very small amount of time, consequently putting you over your usage quota.

I haven't come across any solutions that address this issue. Unless I'm missing something...?

Relevant usage limits for google maps javascript api here.

The link that you posted that says you shouldn't embed API keys in code is related to Google's Cloud Platform. You are fine to leave your API key for Google Maps in your code.