{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-sdk-java/issues/416","labels_url":"https://api.github.com/repos/aws/aws-sdk-java/issues/416/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-sdk-java/issues/416/comments","events_url":"https://api.github.com/repos/aws/aws-sdk-java/issues/416/events","html_url":"https://github.com/aws/aws-sdk-java/issues/416","id":71209933,"number":416,"title":"Cant get MD5 checksum for a multipart upload using Transfer manager","user":{"login":"yagyesha","id":6427944,"avatar_url":"https://avatars.githubusercontent.com/u/6427944?v=3","gravatar_id":"","url":"https://api.github.com/users/yagyesha","html_url":"https://github.com/yagyesha","followers_url":"https://api.github.com/users/yagyesha/followers","following_url":"https://api.github.com/users/yagyesha/following{/other_user}","gists_url":"https://api.github.com/users/yagyesha/gists{/gist_id}","starred_url":"https://api.github.com/users/yagyesha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yagyesha/subscriptions","organizations_url":"https://api.github.com/users/yagyesha/orgs","repos_url":"https://api.github.com/users/yagyesha/repos","events_url":"https://api.github.com/users/yagyesha/events{/privacy}","received_events_url":"https://api.github.com/users/yagyesha/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-04-27T08:01:02Z","updated_at":"2015-04-27T18:00:24Z","closed_at":null,"body":"I have configured TransferManager as below:\r\n\r\n```java\r\nTransferManager tx = new TransferManager(s3, thPool, true);\r\n\r\nTransferManagerConfiguration txConf = new TransferManagerConfiguration();\r\ntxConf.setMinimumUploadPartSize(6*1024*1024); //6MB\r\ntxConf.setMultipartUploadThreshold(10*1024*1024);  //10MB\r\n\r\ntx.setConfiguration(txConf);\r\n```\r\n\r\nRequest object & its metadata is set as below:\r\n\r\n```java\r\n\r\nObjectMetadata metaData = new ObjectMetadata();\r\nmetaData.setContentMD5(streamMD5);\r\n\r\nPutObjectRequest request = new PutObjectRequest(\r\n\t\t\t\t\t\tbucketName, fileToUpload.getName(), fileToUpload)\r\n\t\t\t\t.withMetadata(metaData)\r\n\t\t\t\t.withGeneralProgressListener(progressListener);\r\n```\r\n\r\nAnd finally, i start uploading using\r\n\r\n```java\r\n\r\nUpload upload = tx.upload(request);\r\n```\r\n\r\nAfter upload is completed when i try to get the result and print MD5, it returns null. I only get proper value for ETag (i.e. f45566596bfdfc89264b4bd09a99c4bb-9). But i cannot use it since it is not MD5 checksum.\r\n```java\r\n\r\nUploadResult result = upload.waitForUploadResult();\r\nSystem.out.println(result.getETag());\t\t\t\r\nS3Object uploadObject = s3.getObject(bucketName, fileToUpload.getName());\r\nObjectMetadata objectMetaData = uploadObject.getObjectMetadata();\r\nSystem.out.println(objectMetaData.getETag());\r\nSystem.out.println(objectMetaData.getContentMD5());\r\n```\r\n\r\nAbility to verify the integrity of the uploaded objects is critical for my project as the objects are in GB's & will be downloaded by end customers. \r\n\r\nI'm using aws-java-sdk-s3-1.9.0.\r\nEdit: Also tried with v1.9.6 and the issue is same.\r\n\r\nPlease let me know if i'm doing anything wrong here or is this an issue with SDK. "},"comment":{"url":"https://api.github.com/repos/aws/aws-sdk-java/issues/comments/96760847","html_url":"https://github.com/aws/aws-sdk-java/issues/416#issuecomment-96760847","issue_url":"https://api.github.com/repos/aws/aws-sdk-java/issues/416","id":96760847,"user":{"login":"manikandanrs","id":2710935,"avatar_url":"https://avatars.githubusercontent.com/u/2710935?v=3","gravatar_id":"","url":"https://api.github.com/users/manikandanrs","html_url":"https://github.com/manikandanrs","followers_url":"https://api.github.com/users/manikandanrs/followers","following_url":"https://api.github.com/users/manikandanrs/following{/other_user}","gists_url":"https://api.github.com/users/manikandanrs/gists{/gist_id}","starred_url":"https://api.github.com/users/manikandanrs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/manikandanrs/subscriptions","organizations_url":"https://api.github.com/users/manikandanrs/orgs","repos_url":"https://api.github.com/users/manikandanrs/repos","events_url":"https://api.github.com/users/manikandanrs/events{/privacy}","received_events_url":"https://api.github.com/users/manikandanrs/received_events","type":"User","site_admin":false},"created_at":"2015-04-27T18:00:23Z","updated_at":"2015-04-27T18:00:23Z","body":"TransferManager internally uses the AmazonS3 client to perform the upload and download operations.\r\n\r\nThe uploadPart in AmazonS3Client doesn't set this header if its not available in the UploadPartRequest. <a href=\"https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2764\">Here</a> is the piece of code which does this. \r\n\r\nWhen you use TransferManager to upload objects, it creates new upload part requests for every part and uses the underlying AmazonS3Client. This is <a href=\"https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/transfer/internal/UploadPartRequestFactory.java#L64\">where</a> it creates new UploadPartRequests and as you can see the MD5 is not set.\r\n\r\nHence when you use TransferManager to upload big objects, you don't see this header being set in the request.\r\n\r\nHowever, <a href=\"https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2817\">AmazonS3Client#doUploadPart</a> calculates the MD5 of the individual part as it streams the contents and compares it with the value returned in Etag in the response. The piece of code which does the integrity check is <a href=\"https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java#L2828\">here</a>\r\n\r\nWe will need to fix the uploadPart method in AmazonS3Client to set this header if the source is a file. If the source is a stream (for cases like TransferManager), the check is done at the client side by the Java SDK and not at the server side by S3.\r\n\r\n\r\n"}}