Are refresh token necessary?

I think you are forgetting something.

Refresh tokens are stored on the server. Access tokens are not. Access tokens are self contained. This is why they are referred to as bearer tokens. The bearer of the token is granted access.

Which means if an access token is stolen by a malicious party, they can be used as long it has not expired. Access tokens are considers safe because of their limited life span.

In order to use a refresh token in order to request a new access token. You need to have the client id, client secrete that was used to cerate it. You also need to be able to listen to one of the valid redirect uri's for the refresh token response.