How to set timeout for http.Get() requests in Golang?
I'm making a URL fetcher in Go and have a list of URLs to fetch. I send http.Get()
requests to each URL and obtain their response.
resp,fetch_err := http.Get(url)
How can I set a custom timeout for each Get request? (The default time is very long and that makes my fetcher really slow.) I want my fetcher to have a timeout of around 40-45 seconds after which it should return "request timed out" and move on to the next URL.
How can I achieve this?
Apparently in Go 1.3 http.Client has Timeout field
client := http.Client{
Timeout: 5 * time.Second,
}
client.Get(url)
That's done the trick for me.
You need to set up your own Client with your own Transport which uses a custom Dial function which wraps around DialTimeout.
Something like (completely untested) this:
var timeout = time.Duration(2 * time.Second)
func dialTimeout(network, addr string) (net.Conn, error) {
return net.DialTimeout(network, addr, timeout)
}
func main() {
transport := http.Transport{
Dial: dialTimeout,
}
client := http.Client{
Transport: &transport,
}
resp, err := client.Get("http://some.url")
}