Azure ARM template - ssh public and private key

Yes, you should be using Azure Key Vault for that. You can provide secure parameter to a VM at deployment time, like so:

"osProfile": {
  "computerName": "[parameters('vmName')]",
  "adminUsername": "[parameters('adminUsername')]",
  "adminPassword": "[parameters('adminPassword')]",
  "secrets": [
    {
      "sourceVault": {
        "id": "[resourceId(parameters('vaultResourceGroup'), 'Microsoft.KeyVault/vaults', parameters('vaultName'))]"
      },
      "vaultCertificates": [
        {
          "certificateUrl": "[parameters('certificateUrl')]",
          "certificateStore": "My"
        }
      ]
    }
  ],

but for Linux you should use something like this:

"osProfile": {
                "linuxOperatingSystemProfile": {
                  "username": "[parameters('sshUserName')]",
                  "sshProfile": {
                    "publicKeys": [
                      {
                        "certificateData": "[parameters('sshPublicKey')]"
                      }
                    ]
                  }
                }
              }

but, obviously substitute certificateData with ssh key in your key vault.