HttpWebRequest get the same response object when it's called in the second time
<p>I am using HttpWebRequest to get webpage source code with POST method.
The page needs to be accessed for multiple times with different parameters.</p>

<p>When getting response in the second time in a short period, it always return the same response object to me. After debugging, if the second call is after around 30 seconds, the request can get the correct response object(source code).</p>

<pre><code>public HtmlAgilityPack.HtmlDocument getHtmlData(string url, string cname)
    {
        HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();

        try
        {
            HttpRequestCachePolicy policy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);
            HttpWebRequest.DefaultCachePolicy = policy;
            HttpRequestCachePolicy noCachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);

            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
            req.Method = "POST";
            req.ContentType = "application/x-www-form-urlencoded";
            req.KeepAlive = false;
            req.Headers[HttpRequestHeader.CacheControl] = "no-cache";
            req.Headers[HttpRequestHeader.Pragma] = "no-cache";
            req.IfModifiedSince = DateTime.Now;
            req.CachePolicy = noCachePolicy;

            //add post data
            string postData = "cname=" + cname;
            byte[] byteArray = Encoding.GetEncoding("shift-jis").GetBytes(postData);
            req.ContentLength = byteArray.Length;
            using (Stream stream = req.GetRequestStream())
            {
                stream.Write(byteArray, 0, byteArray.Length);
            }

            // get response
            string data = "";
            HttpWebResponse response = req.GetResponse() as HttpWebResponse;
            using (var stream = response.GetResponseStream())
            {
                StreamReader reader = new StreamReader(stream, Encoding.GetEncoding("shift-jis"));
                data = reader.ReadToEnd();
                reader.Close();
            }

            response.Close();
            doc.LoadHtml(data);
            //System.Threading.Thread.Sleep(30000);
        }
        catch(Exception ex)
        {
            Log.log.Debug(ex.ToString());
        }

        return doc;
    }
</code></pre>

<p>Function call block:</p>

<pre><code>getHtmlData(@"http://www.jra.go.jp/JRADB/accessS.html", "pw01sli00/AF");
getHtmlData(@"http://www.jra.go.jp/JRADB/accessS.html", "pw01skl00999999/B3");
getHtmlData(@"http://www.jra.go.jp/JRADB/accessS.html", "pw01skl00201604/E3");
</code></pre>

<p>I have stuck with the problem for a whole day. hope someone can give me a clue.</p>

<p>Many thanks!</p>