Trying to consume HUGE JSON data using Spring RestTemplate resulting in java heap space exception
<p>I am using Resttemplate to consume a huge json from an external server. My code works fine when the dataset is not huge but as soon as I run it against the full dataset, I am unable to map the response to the bean class. Below is my code.</p>

<blockquote>
  <p></p>
</blockquote>

<pre><code>public void createCustomCsv(String name, String password, String serverUrl1, String serverUrl2, String propLocation)
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);

RestTemplate restTemplate = new RestTemplate();
restTemplate.setRequestFactory(requestFactory);

HttpHeaders httpHeaders = customHeaders.createCustomHeaders(name, password);

List&lt;HttpMessageConverter&lt;?&gt;&gt; messageConverters = new ArrayList&lt;HttpMessageConverter&lt;?&gt;&gt;();
 //Add the required converters
messageConverters.add(new MappingJackson2HttpMessageConverter());
messageConverters.add(new StringHttpMessageConverter());
 //Add the message converters to the restTemplate
restTemplate.setMessageConverters(messageConverters); 

ResponseEntity&lt;MyDataBean&gt; responseEntity1;
ResponseEntity&lt;MyDataBean&gt; responseEntity2;
try {
    long startTime = System.currentTimeMillis();
    jLog.debug("Start mapping to Pojo :: " +startTime);
    responseEntity1 = restTemplate.exchange(serverUrl1, HttpMethod.GET, httpEntity, MyDataBean.class);
 responseEntity2 = restTemplate.exchange(serverUrl2, HttpMethod.GET, httpEntity, MyDataBean.class);


MyDataBean sampleDataBeanServer1 = responseEntity1 .getBody();
MyDataBean sampleDataBeanServer2 = responseEntity2 .getBody();


processCustomData(sampleDataBeanServer1 , propLocation);
processCustomData(sampleDataBeanServer2 , propLocation);
} catch (RestClientException e) {

    jLog.debug("Something went wrong with ws call:::" +e);
}
}
</code></pre>

<blockquote>
  <p></p>
</blockquote>

<p>I have tried for hours and searched in SO trying to figure out a way to parse the response data into the Pojo class without getting memory exceptions. I understand that this might be because the entire response is being retained in memory to map it to the Pojo. The below code is where the memory problem is found. </p>

<pre><code> responseEntity1 = restTemplate.exchange(serverUrl1, HttpMethod.GET, httpEntity, MyDataBean.class); 
</code></pre>

<p>Based on docs, I have also set the BufferRequestBody to false so that the response doesn't get loaded into memory. I am not sure why the behaviour is different as compared to the docs.</p>

<pre><code> SimpleClientHttpRequestFactory requestFactory = new  SimpleClientHttpRequestFactory();
 requestFactory.setBufferRequestBody(false);
</code></pre>

<p>It would be great if someone is patient and kind to help me out of this soup! I am open to any third party library suggestions as well. </p>

<p><strong>P.N:</strong> The above code works perfect when dataset is not huge. Also omitted the bean class code for brevity.</p>