HttpUrlConnection to get content from aspx?
<p>So, I want to get data from aspx page. And I need to do the follow steps</p>

<ol>1)Send GET request and get __VIEWSTATE and __EVENTVALIDATION.</ol>

<ol>2)Send POST request with parameters (to login) then update __VIEWSTATE and __EVENTVALIDATION</ol>

<ol>3)Send POST request with parameters (to choose field) then update variables too</ol>

<ol>4)Send POST request with parameters (to press button) </ol>

<ol>5)Parse page content.</ol>

<p>Page content dynamically loaded after script.
I tryed to use HttpUrlConnection</p>

<pre><code> public String sendGet(String url) throws Exception {
        StringBuilder result = new StringBuilder();
        URL ur = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();
        connection.setRequestMethod("GET");
        connection.setUseCaches(false);
        connection.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36");
        connection.setRequestProperty("Accept",
                "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
        connection.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String inputLine;
        while ((inputLine = br.readLine()) != null) {
            result.append(inputLine);
        }
        br.close();

        return result.toString();
    }
 public String sendPost(String url, List&lt;NameValuePair&gt; formParams) throws Exception {
        URL ur = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();
        connection.setReadTimeout(10000);
        connection.setDoOutput(true);
        connection.setDoInput(true);
        connection.setConnectTimeout(10000);
        connection.setRequestMethod("POST");
        for (HttpCookie cookie : this.cookies) {
            connection.addRequestProperty("Cookie", cookie.toString());
        }
        connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36");
        connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
        connection.setRequestProperty("Content-Language", "en-US");
        connection.connect();
        OutputStream os = connection.getOutputStream();
        BufferedWriter writer = new BufferedWriter(
                new OutputStreamWriter(os, "UTF-8"));
        writer.write(getQuery(formParams));
        writer.flush();
        writer.close();
        os.close();
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String response = "";
        String line;
        while ((line = br.readLine()) != null) {
            response += line;
        }
        System.out.println(Jsoup.parse(response).text());
        updateViewState(response);
        updateSubSession(response);
        return response;
    }
</code></pre>

<p>And I stayed in the second step. POST and GET subSession is different , but Cookie is the same :). So, EVENTVALIDATION and VIEWSTATE is invalid, and I get "login.aspx" page again. This is how I get Cookie    </p>

<pre><code>List&lt;HttpCookie&gt; cookies;
public void init() throws Exception {
        CookieManager manager = new CookieManager();
        manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
        CookieHandler.setDefault(manager);
        String html = sendGet(url);
        updateViewState(html);
        updateSubSession(html);
        CookieStore cookieJar = manager.getCookieStore();
        cookies=cookieJar.getCookies();
    }
</code></pre>

<p>So, I tryed to send POST with subSession.</p>

<pre><code>public String sendPost(String url, List&lt;NameValuePair&gt; formParams) throws Exception {
        URL ur = new URL(url+subSession);
        HttpURLConnection connection = (HttpURLConnection) ur.openConnection();
</code></pre>

<p>And...</p>

<pre><code>Exception in thread "main" java.io.IOException: Server returned HTTP response code: 500 for URL: http://www.aogc2.state.ar.us:8080/DWClient/Login.aspx?DWSubSession=9852&amp;v=1589
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
    at com.company.UrlConnection.sendPost(UrlConnection.java:111)
    at com.company.UrlConnection.login(UrlConnection.java:138)
    at com.company.UrlConnection.start(UrlConnection.java:35)
    at com.company.Main.main(Main.java:11)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at 
</code></pre>

<p>I thought that I was wrong to get cookie and I tryed to get cookie like this:</p>

<pre><code>connection.getHeaderFields().get("Set-Cookie");  
</code></pre>

<p>but    <code>Cookie</code>     were     <code>null</code>    .
Where I was wrong? Thank you a lot!</p>