API call getting timed out for large amout of data
<p>I am having trouble in fetching XML data from through API in large amount. Has anyone a better solution in fetching XML data in ASP.net application?</p>

<p>Maybe the issue is might be garbage collection memory not sure but earlier it was not working for any user so I added manual memory release code and closing web response connection after each call. </p>

<p>Any suggestion to improve the fetching of large amount of data are welcome?</p>

<pre><code>   public static XmlDocument LoadXmlFromUrl(string url, bool keepAlive = true, bool throwException = false, int timeout = 15000)
    {
        Log(url, "*** start ***" + url);

        ServicePointManager.DefaultConnectionLimit = 2000;
        string xmlAsText = null;
        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
        req.KeepAlive = keepAlive;
        req.Timeout = timeout; // 30,000ms = 30 sec default
        try
        {
            bool retried = false;
        retry:
            try
            {
                using (HttpWebResponse webResponse = (HttpWebResponse)req.GetResponse())
                {
                    using (StreamReader responseStream = new StreamReader(webResponse.GetResponseStream(), Encoding.UTF8))
                    {
                        xmlAsText = responseStream.ReadToEnd();
                        responseStream.Close();
                    }
                    webResponse.Close();

                    Log(url, "stream reader (responseStream.ReadToEnd) completed to string.");
                }
            }
            catch (WebException webEx)
            {
                if (!retried)
                {
                    if (webEx.Status == WebExceptionStatus.Timeout || webEx.Status == WebExceptionStatus.KeepAliveFailure)
                    {
                        Log(url, "About to retry for url (6 sec wait): " + url);
                        System.Threading.Thread.Sleep(6000);
                        retried = true;
                        goto retry;
                    }
                }
                else
                {
                    throw webEx;
                }
            }
        }
        catch (Exception ex)
        {
            Log(url, "exception while loading responseStream.ReadToEnd. " + ex.ToString());
        }
        XmlDocument xdoc = null;
        if (!string.IsNullOrEmpty(xmlAsText))
        {
            try
            {
                xdoc = new XmlDocument();
                xdoc.LoadXml(xmlAsText);
                Log(url, "string added to xml (xdoc.LoadXml) completed.");
            }
            catch (Exception ex)
            {
                Log(url, "exception while loading xdoc.LoadXml. " + ex.ToString());
            }
        }
        return xdoc;
    }
</code></pre>

<p>Exception 1:</p>

<pre><code>5/01/2017 12:27:01 p.m. exception while loading responseStream.ReadToEnd.     System.Net.WebException: The operation has timed out**
</code></pre>

<p>Exception 2:</p>

<pre><code>6/01/2017 8:35:59 a.m. exception while loading responseStream.ReadToEnd. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
</code></pre>