How to do a https request with bad certificate?

Say I want to get https://golang.org programatically. Currently golang.org (ssl) has a bad certificate which is issued to *.appspot.com So when I run this:

package main

import (
    "log"
    "net/http"
)

func main() {
    _, err := http.Get("https://golang.org/")
    if err != nil {
        log.Fatal(err)
    }
}

I get (as I expected)

Get https://golang.org/: certificate is valid for *.appspot.com, *.*.appspot.com, appspot.com, not golang.org

Now, I want to trust this certificate myself (imagine a self-issued certificate where I can validate fingerprint etc.): how can I make a request and validate/trust the certificate?

I probably need to use openssl to download the certificate, load it into my file and fill tls.Config struct !?

Security note: Disabling security checks is dangerous and should be avoided

You can disable security checks globally for all requests of the default client:

package main

import (
    "fmt"
    "net/http"
    "crypto/tls"
)

func main() {
    http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
    _, err := http.Get("https://golang.org/")
    if err != nil {
        fmt.Println(err)
    }
}

You can disable security check for a client:

package main

import (
    "fmt"
    "net/http"
    "crypto/tls"
)

func main() {
    tr := &http.Transport{
        TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
    }
    client := &http.Client{Transport: tr}
    _, err := client.Get("https://golang.org/")
    if err != nil {
        fmt.Println(err)
    }
}

Proper way (as of Go 1.13) (provided by answer below):

customTransport := http.DefaultTransport.(*http.Transport).Clone()
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: customTransport}

Original Answer:

Here's a way to do it without losing the default settings of the DefaultTransport, and without needing the fake request as per user comment.

defaultTransport := http.DefaultTransport.(*http.Transport)

// Create new Transport that ignores self-signed SSL
customTransport := &http.Transport{
  Proxy:                 defaultTransport.Proxy,
  DialContext:           defaultTransport.DialContext,
  MaxIdleConns:          defaultTransport.MaxIdleConns,
  IdleConnTimeout:       defaultTransport.IdleConnTimeout,
  ExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,
  TLSHandshakeTimeout:   defaultTransport.TLSHandshakeTimeout,
  TLSClientConfig:       &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: customTransport}

Shorter way:

customTransport := &(*http.DefaultTransport.(*http.Transport)) // make shallow copy
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: customTransport}

Warning: For testing/development purposes only. Anything else, proceed at your own risk!!!

All of these answers are wrong! Do not use InsecureSkipVerify to deal with a CN that doesn't match the hostname. The Go developers unwisely were adamant about not disabling hostname checks (which has legitimate uses - tunnels, nats, shared cluster certs, etc), while also having something that looks similar but actually completely ignores the certificate check. You need to know that the certificate is valid and signed by a cert that you trust. But in common scenarios, you know that the CN won't match the hostname you connected with. For those, set ServerName on tls.Config. If tls.Config.ServerName == remoteServerCN, then the certificate check will succeed. This is what you want. InsecureSkipVerify means that there is NO authentication; and it's ripe for a Man-In-The-Middle; defeating the purpose of using TLS.

There is one legitimate use for InsecureSkipVerify: use it to connect to a host and grab its certificate, then immediately disconnect. If you setup your code to use InsecureSkipVerify, it's generally because you didn't set ServerName properly (it will need to come from an env var or something - don't belly-ache about this requirement... do it correctly).

In particular, if you use client certs and rely on them for authentication, you basically have a fake login that doesn't actually login any more. Refuse code that does InsecureSkipVerify, or you will learn what is wrong with it the hard way!

The correct way to do this if you want to maintain the default transport settings is now (as of Go 1.13):

customTransport := http.DefaultTransport.(*http.Transport).Clone()
customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client = &http.Client{Transport: customTransport}

Transport.Clone makes a deep copy of the transport. This way you don't have to worry about missing any new fields that get added to the Transport struct over time.

If you want to use the default settings from http package, so you don't need to create a new Transport and Client object, you can change to ignore the certificate verification like this:

tr := http.DefaultTransport.(*http.Transport)
tr.TLSClientConfig.InsecureSkipVerify = true

How to do a https request with bad certificate?

Related

Recent Posts