The operation has timed out at System.Net.HttpWebRequest.GetResponse() while sending large number of requests to a host
I am sending a large number of simultaneous requests to a particular web service with different data. To achieve this, I have created a number of threads(around 50 in number). Total number of requests per minute may increase up to 10000. The application in the form of a windows service runs fine for a few minutes and then a operation time out error is encountered.
I have tried the usual suspects such as increasing DefaultConnectionLimit, closing the web response object. Since the requests do not take much time on server, I have also set the request Timeout and ReadWriteTimeout to 5 seconds. Below is the code snippet which is called repeatedly by different threads.
// Below line is executed at the start of application
ServicePointManager.DefaultConnectionLimit = 15000;
// Below code is executed at repeatedly by different threads
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Host = hostName;
request.Proxy = null;
request.UserAgent = "Windows Service";
byte[] bytes = new byte[0];
if (body != null)
{
bytes = System.Text.Encoding.ASCII.GetBytes(body);
request.ContentType = "text/xml; encoding='utf-8'";
request.ContentLength = bytes.Length;
}
request.Method = "POST";
request.Timeout = 5000;
request.ReadWriteTimeout = 5000;
request.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes(username + ":" + password));
request.CookieContainer = this.cookieContainer;
if (body != null)
{
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
}
HttpWebResponse httpResponse = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
responseText = streamReader.ReadToEnd();
}
httpResponse.Close();
ServicePointManager.DefaultConnectionLimit
limits the number of outgoing web requests to a given server. The default is generally 2 or 10.
If you are making 50 parallel calls to that web service, you should set ServicePointManager.DefaultConnectionLimit
(at app startup) to a larger number (e.g. 40-50).
Additionally, you are not calling Close
or Dispose
on request
. You should do this, or let using
take care of it for you.