is it possible to setup VPN using external IP(Public) of server through Google Cloud VPN

Unfortunately it's not possible to use Cloud VPN to connect to external IP of your VM instance.

Have a look at the documentation Cloud VPN:

Cloud VPN securely connects your peer network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection.Traffic traveling between the two networks is encrypted by one VPN gateway, and then decrypted by the other VPN gateway.

and at the section Specifications:

Cloud VPN only supports site-to-site IPsec VPN connectivity, subject to the requirements listed in this section. It does not support client-to-gateway (road warrior) scenarios.

also, at the documentation Networks and tunnel routing section Supported networks:

Cloud VPN supports VPC custom networks, auto-mode networks, and legacy networks;

In addition, at the documentation IP Addresses:

Each VM instance can have one primary internal IP address, one or more secondary IP addresses, and one external IP address. To communicate between instances on the same Virtual Private Cloud (VPC) network, you can use the internal IP address for the instance. To communicate with the internet, you must use the instance's external IP address unless you have configured a proxy of some kind. Similarly, you must use the instance's external IP address to connect to instances outside of the same VPC network unless the networks are connected in some way, like via Cloud VPN.

As a result, you'll be able to connect from your on-premises network to internal IP of your server only while using Cloud VPN.

As a workaround, you can create a firewall rule to allow traffic from remote public IP at GCP Firewall if it's possible in your use case. In some scenarios, you can use Cloud IAP for secure access to GCP VM instance via SSH or RDP.