net/http: request canceled (Client.Timeout exceeded while awaiting headers) why/what to do with this?
<p>TL:DR - Look at edit 2 for a C# equivalent of code http client code, resulting in ~same issue, so Go http.Client is not the real problem, but C# Web API once deployed to Azure...</p>

<p>I'm getting very bad performance from a C# Web API once deployed to Azure Web App [2x Standard S3]. At first I was asking re: Go's http.Client timeout, but writing a similar client in C# and NodeJs gave same results.</p>

<p>This is my http.Client:</p>

<pre><code>func getWebClient() *http.Client {
    var netTransport = &amp;http.Transport{
        Dial: (&amp;net.Dialer{
            Timeout: 5 * time.Second,
        }).Dial,
        TLSHandshakeTimeout:   10 * time.Second,
        MaxIdleConnsPerHost:   2000,
        ResponseHeaderTimeout: 10 * time.Second,
    }

    var netClient = &amp;http.Client{
        Timeout:   time.Second * 10,
        Transport: netTransport,
    }

    return netClient
}
</code></pre>

<p>Error I'm getting:</p>

<p><code>net/http: request canceled (Client.Timeout exceeded while awaiting headers)</code></p>

<p>It can be on GET, POST, PUT. I'm getting those error, yet running a failing GET with curl immediately got a reply.</p>

<p>This is a sample Get function I'm using to call an API:</p>

<pre><code>func get(path string, result interface{}) error {
    req, err := http.NewRequest("GET", webDALAPIURL+path, nil)
    if err != nil {
        return err
    }

    req.Header.Set("Content-Type", "application/json")

    wc := getWebClient()
    res, err := wc.Do(req)
    if err != nil {
        return err
    }
    defer res.Body.Close()

    if res.StatusCode &gt;= 400 {
        return fmt.Errorf("[GET] %d - %s", res.StatusCode, webDALAPIURL+path)
    }

    decoder := json.NewDecoder(res.Body)
    return decoder.Decode(result)
}
</code></pre>

<p>Fun fact, never encountered those error when the API was running locally. API is a C# ASP.NET Web API app. </p>

<p>I started to have lots of TLS Handshake error, so I dropped the https for the Azure app endpoint. Now I'm getting this error.</p>

<p>I'm tailing the logs for the app, and nothing is occurring [the API being called]. Seems like Go is unable to make multiple call to same host. I'm not event using goroutine in one cmd and using them in another, both results in same errors.</p>

<p>When the API is running on a Windows computer in same network, never had that error during development.</p>

<p>Edit 1:</p>

<p>Note that 80-85% of the requests are working well, the scenario is that (pseudo code):</p>

<pre><code>for item in items {
  http get /item/{id} =&gt; works 90% of the time, 10% timeout

  change item properties

  http PUT /item/{id} =&gt; works 80% of the time
}
</code></pre>

<p>I added a retry in the <code>get()</code> function, so if the timeout occurs it's retrying the get, and this seems to works. Although, I don't like this  workaround at all.</p>

<p>Also note that we are talking about quick GET that are returning timeout, when I run them from curl, it's &lt; 1sec. Same for the PUT, those are highly simplistic database SELECT * FROM TABLE WHERE ID = {id} AND UPDATE.</p>

<p>The Azure Web app that runs the API is 2 instance of Standard S3.</p>

<p>The fact that the retry for the GET worked looks like, it's most certainly the API / Azure app not taking the load, which is impossible for the simplicity and we are talking about less than 10 requests / seconds.</p>

<p>Another point not negligible, when running on dev server, the same Azure SQL Database was used, so the SELECT/UPDATE perf should be the exact same on dev and on Azure Web App. </p>

<p><strong>Edit 2:</strong></p>

<p>The speed comparing same C# Web API from local to Azure is disturbing. I wrote a similar C# http client to test Azure vs Local Web API.</p>

<pre><code>class Program
{
  static int fails = 0;
  static void Main(string[] args)
  {
    for (int i = 0; i &lt; 2000; i++)
    {
      get(i);
    }
    Console.WriteLine("completed: " + fails.ToString());
    Console.ReadLine();
  }
  static void get(int id)
  {
    id += 22700;
    var c = new HttpClient();

    var resp = c.GetAsync("http://myapphere.azurewebsites.net/api/users/" + id).Result;
    if (!resp.IsSuccessStatusCode)
    {
      Console.WriteLine("");
      fails++;
      Console.WriteLine(string.Format("error getting /users status code {0}", resp.StatusCode));
    }
    else
    {
      Console.Write(".");
    }
  }
}
</code></pre>

<p>Running this console app <strong>against Azure</strong> I can clearly see where Go's getting time out, it's painfully slow, no error are returned, but the Console.Write("."); takes forever to print, are are periodic, can print ~fast for 3-4 than it stopped.</p>

<p>Changing this <strong>to localhost:1078</strong> again <strong>same database used</strong> there's no paused, and the Console.Write(".") are printing like 20x the speed compare to Azure.</p>

<p>How is that possible?</p>

<p>Edit 3:</p>

<p>Just added global error handler on the web api, maybe the slovenliness could be caused by too much Exception being thrown. Added a <code>Trace.TraceError</code> and I <code>azure site log tail</code> it, again nothing was displayed.</p>

<p>I would go as far as saying that the local web api is running 25-30x faster than the 2x instance of Azure Standard S3. Clearly that can't be true, but the web api is so simple that I don't see what I can do to have Azure run it at full speed.</p>