Using AWS Lambda to transform files on S3
<p>I have a very simple scenario. 
I use Lambda to load multiple 100MB files from S3 (every 1-10 seconds a new file(s) land on s3), transform them (minor changes) and store them on a different S3 Bucket. </p>

<p>The problem in my Java program (running on AWS Lambda) is giving Memory overflow errors even though I allocated 1024MD RAM. I load the complete files in RAM. Not sue why this happens.</p>

<p>I have following questions:</p>

<ol>
<li>Without filling my RAM, what is a good approach to read, transform and write 100MB files on S3 repeatedly and concurrently? Is Lambda a good choice for this case?</li>
<li>Why is my memory getting full even though I allocated 1024 memory through web UI? </li>
</ol>