C# HttpWebRequest.GetResponse() 不断超时

HttpWebRequest.GetResponse() keeps getting timed out

提问by symbiont

i wrote a simple C# function to retrieve trade history from MtGox with following API call:

我编写了一个简单的 C# 函数，通过以下 API 调用从 MtGox 检索交易历史：

https://data.mtgox.com/api/1/BTCUSD/trades?since=<trade_id>

documented here: https://en.bitcoin.it/wiki/MtGox/API/HTTP/v1#Multi_currency_trades

此处记录：https: //en.bitcoin.it/wiki/MtGox/API/HTTP/v1#Multi_currency_trades

here's the function:

这是功能：

string GetTradesOnline(Int64 tid)
{
    Thread.Sleep(30000);

    // communicate
    string url = "https://data.mtgox.com/api/1/BTCUSD/trades?since=" + tid.ToString();
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    StreamReader reader = new StreamReader(response.GetResponseStream());

    string json = reader.ReadToEnd();
    reader.Close();
    reader.Dispose();
    response.Close();

    return json;
}

i'm starting at tid=0 (trade id) to get the data (from the very beginning). for each request, i receive a response containing 1000 trade details. i always send the trade id from the previous response for the next request. it works fine for exactly 4 requests & responses. but after that, the following line throws a "System.Net.WebException", saying that "The operation has timed out":

我从 tid=0 (trade id) 开始获取数据（从一开始）。对于每个请求，我都会收到一个包含 1000 个交易详细信息的响应。对于下一个请求，我总是从上一个响应中发送交易 ID。它正好适用于 4 个请求和响应。但在那之后，以下行抛出“System.Net.WebException”，表示“操作已超时”：

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

here are the facts:

以下是事实：

• catching the exception and retying keeps causing the same exception
• the default HttpWebRequest .Timeout and .ReadWriteTimeout are already high enough (over a minute)
• changing HttpWebRequest.KeepAlive to false didn't solve anything either
• it seems to always work in the browser even while the function is failing
• it has no problems retrieveing the response from https://www.google.com
• the amount of successful responses before the exceptions varies from day to day (but browser always works)
• starting at the trade id that failed last time causes the exception immediately
• calling this function from the main thread instead still caused the exception
• running it on a different machine didn't work
• running it from a different IP didn't work
• increasing Thread.Sleep inbetween requests does not help

• 捕获异常并重新绑定不断导致相同的异常
• 默认的 HttpWebRequest .Timeout 和 .ReadWriteTimeout 已经足够高（超过一分钟）
• 将 HttpWebRequest.KeepAlive 更改为 false 也没有解决任何问题
• 即使功能失败，它似乎总是在浏览器中工作
• 从https://www.google.com检索响应没有问题
• 异常之前的成功响应数量每天都在变化（但浏览器始终有效）
• 从上次失败的交易 ID 开始立即导致异常
• 从主线程调用此函数仍然导致异常
• 在另一台机器上运行它不起作用
• 从不同的 IP 运行它不起作用
• 在请求之间增加 Thread.Sleep 没有帮助

any ideas of what could be wrong?

关于什么可能是错误的任何想法？

采纳答案by Habeeb

I had the very same issue. For me the fix was as simple as wrapping the HttpWebResponse code in using block.

我有同样的问题。对我来说，修复就像在 using 块中包装 HttpWebResponse 代码一样简单。

using (HttpWebResponse response = (HttpWebResponse) request.GetResponse())
{
    // Do your processings here....
}

Details: This issue usually happens when several requests are made to the same host, and WebResponseis not disposed properly. That is where usingblock will properly dispose the WebResponseobject properly and thus solving the issue.

详细信息：此问题通常发生在向同一主机发出多个请求WebResponse且未正确处理时。这就是usingblock 将正确处理WebResponse对象并从而解决问题的地方。

回答by kevin c

There are two kind of timeouts. Client timeout and server timeout. Have you tried doing something like this:

有两种超时。客户端超时和服务器超时。你有没有试过做这样的事情：

request.Timeout = Timeout.Infinite;
request.KeepAlive = true;

Try something like this...

尝试这样的事情......

回答by Karl

I just had similar troubles calling a REST Service on a LINUX Server thru ssl. After trying many different configuration scenarios I found out that I had to send a UserAgent in the http head.

我只是在通过 ssl 在 LINUX 服务器上调用 REST 服务时遇到了类似的麻烦。在尝试了许多不同的配置方案后，我发现我必须在 http 头中发送一个 UserAgent。

Here is my final method for calling the REST API.

这是我调用 REST API 的最终方法。

        private static string RunWebRequest(string url, string json)
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

        // Header
        request.ContentType = "application/json";
        request.Method = "POST";
        request.AllowAutoRedirect = false;
        request.KeepAlive = false;
        request.Timeout = 30000;
        request.ReadWriteTimeout = 30000;
        request.UserAgent = "test.net";
        request.Accept = "application/json";
        request.ProtocolVersion = HttpVersion.Version11;
        request.Headers.Add("Accept-Language","de_DE");
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
        ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
        byte[] bytes = Encoding.UTF8.GetBytes(json);
        request.ContentLength = bytes.Length;
        using (var writer = request.GetRequestStream())
        {
            writer.Write(bytes, 0, bytes.Length);
            writer.Flush();
            writer.Close();
        }

        var httpResponse = (HttpWebResponse)request.GetResponse();
        using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
        {
            var jsonReturn = streamReader.ReadToEnd();
            return jsonReturn;
        }
    }

回答by bech

For what it's worth, I was experiencing the same issues with timeouts every time I used it, even though calls went through to the server I was calling. The problem in my case was that I had Expect set to application/json, when that wasn't what the server was returning.

对于它的价值，我每次使用它时都会遇到相同的超时问题，即使调用通过了我正在调用的服务器。我的问题是我将 Expect 设置为 application/json，而这不是服务器返回的内容。

回答by symbiont

This is not a solution, but just an alternative: These days i almost only use WebClient instead of HttpWebRequest. Especially WebClient.UploadString for POST and PUT and WebClient.DownloadString. These simply take and return strings. This way i don't have to deal with streams objects, except when i get a WebException. i can also set the content type with WebClient.Headers["Content-type"] if necessary. The using statement also makes life easier by calling Dispose for me.

这不是解决方案，而只是替代方案：这些天我几乎只使用 WebClient 而不是 HttpWebRequest。尤其是用于 POST 和 PUT 的 WebClient.UploadString 和 WebClient.DownloadString。这些只是获取和返回字符串。这样我就不必处理流对象，除非我得到一个 WebException。如有必要，我还可以使用 WebClient.Headers["Content-type"] 设置内容类型。using 语句还通过为我调用 Dispose 使生活更轻松。

Rarely for performance, i set System.Net.ServicePointManager.DefaultConnectionLimit high and instead use HttpClient with it's Async methods for simultaneous calls.

很少为了性能，我将 System.Net.ServicePointManager.DefaultConnectionLimit 设置为高，而是使用 HttpClient 及其异步方法进行同步调用。

This is how i would do it now

这就是我现在要做的

string GetTradesOnline(Int64 tid)
{
    using (var wc = new WebClient())
    {
        return wc.DownloadString("https://data.mtgox.com/api/1/BTCUSD/trades?since=" + tid.ToString());
    }
}

2 more POST examples

还有 2 个 POST 示例

// POST
string SubmitData(string data)
{
    string response;
    using (var wc = new WebClient())
    {
        wc.Headers["Content-type"] = "text/plain";
        response = wc.UploadString("https://data.mtgox.com/api/1/BTCUSD/trades", "POST", data);
    }

    return response;
}

// POST: easily url encode multiple parameters
string SubmitForm(string project, string subject, string sender, string message)
{
    // url encoded query
    NameValueCollection query = HttpUtility.ParseQueryString(string.Empty);
    query.Add("project", project);
    query.Add("subject", subject);

    // url encoded data
    NameValueCollection data = HttpUtility.ParseQueryString(string.Empty);
    data.Add("sender", sender);
    data.Add("message", message);

    string response;
    using (var wc = new WebClient())
    {
        wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
        response = wc.UploadString( "https://data.mtgox.com/api/1/BTCUSD/trades?"+query.ToString()
                                  , WebRequestMethods.Http.Post
                                  , data.ToString()
                                  );
    }

    return response;
}

Error handling

错误处理

try
{
    Console.WriteLine(GetTradesOnline(0));

    string data = File.ReadAllText(@"C:\mydata.txt");
    Console.WriteLine(SubmitData(data));

    Console.WriteLine(SubmitForm("The Big Project", "Progress", "John Smith", "almost done"));
}
catch (WebException ex)
{
    string msg;
    if (ex.Response != null)
    {
        // read response HTTP body
        using (var sr = new StreamReader(ex.Response.GetResponseStream())) msg = sr.ReadToEnd();
    }
    else
    {
        msg = ex.Message;
    }

    Log(msg);
    throw; // re-throw without changing the stack trace
}