{"action":"closed","issue":{"url":"https://api.github.com/repos/leo-project/leofs/issues/385","labels_url":"https://api.github.com/repos/leo-project/leofs/issues/385/labels{/name}","comments_url":"https://api.github.com/repos/leo-project/leofs/issues/385/comments","events_url":"https://api.github.com/repos/leo-project/leofs/issues/385/events","html_url":"https://github.com/leo-project/leofs/issues/385","id":89004651,"number":385,"title":"[leo_object_storage] Find incorrect some data-blocks during the data-compaction","user":{"login":"yosukehara","id":148770,"avatar_url":"https://avatars.githubusercontent.com/u/148770?v=3","gravatar_id":"","url":"https://api.github.com/users/yosukehara","html_url":"https://github.com/yosukehara","followers_url":"https://api.github.com/users/yosukehara/followers","following_url":"https://api.github.com/users/yosukehara/following{/other_user}","gists_url":"https://api.github.com/users/yosukehara/gists{/gist_id}","starred_url":"https://api.github.com/users/yosukehara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yosukehara/subscriptions","organizations_url":"https://api.github.com/users/yosukehara/orgs","repos_url":"https://api.github.com/users/yosukehara/repos","events_url":"https://api.github.com/users/yosukehara/events{/privacy}","received_events_url":"https://api.github.com/users/yosukehara/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/leo-project/leofs/labels/Bug","name":"Bug","color":"fc2929"},{"url":"https://api.github.com/repos/leo-project/leofs/labels/Improve","name":"Improve","color":"FFA500"},{"url":"https://api.github.com/repos/leo-project/leofs/labels/Priority-HIGH","name":"Priority-HIGH","color":"f7c6c7"},{"url":"https://api.github.com/repos/leo-project/leofs/labels/_leo_object_storage","name":"_leo_object_storage","color":"ededed"},{"url":"https://api.github.com/repos/leo-project/leofs/labels/_leo_storage","name":"_leo_storage","color":"ededed"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/leo-project/leofs/milestones/43","html_url":"https://github.com/leo-project/leofs/milestones/1.2.11","labels_url":"https://api.github.com/repos/leo-project/leofs/milestones/43/labels","id":1148852,"number":43,"title":"1.2.11","description":"","creator":{"login":"yosukehara","id":148770,"avatar_url":"https://avatars.githubusercontent.com/u/148770?v=3","gravatar_id":"","url":"https://api.github.com/users/yosukehara","html_url":"https://github.com/yosukehara","followers_url":"https://api.github.com/users/yosukehara/followers","following_url":"https://api.github.com/users/yosukehara/following{/other_user}","gists_url":"https://api.github.com/users/yosukehara/gists{/gist_id}","starred_url":"https://api.github.com/users/yosukehara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yosukehara/subscriptions","organizations_url":"https://api.github.com/users/yosukehara/orgs","repos_url":"https://api.github.com/users/yosukehara/repos","events_url":"https://api.github.com/users/yosukehara/events{/privacy}","received_events_url":"https://api.github.com/users/yosukehara/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":6,"state":"open","created_at":"2015-06-04T07:20:47Z","updated_at":"2015-06-17T12:36:48Z","due_on":"2015-06-16T15:00:00Z","closed_at":null},"comments":1,"created_at":"2015-06-17T12:34:58Z","updated_at":"2015-06-17T12:36:48Z","closed_at":"2015-06-17T12:36:48Z","body":"Last Friday, I found incorrect some data-blocks at some Leo Storage's AVS files as below:\r\n\r\n### Error log\r\n```\r\n[E]\tstorage_0@127.0.0.1\t2015-06-15 00:45:15.130856 +0000\t1434329115\tleo_compact_fsm_worker:execute_1/4\t1036\t[{obj_container_path,\"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/2.avs_63601547870\"},{error_pos_start,577},{error_pos_end,739},{errors,[{invalid_format,unexpected_time_format}]}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-15 00:45:15.132416 +0000\t1434329115\tleo_compact_fsm_worker:execute_1/4\t1036\t[{obj_container_path,\"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870\"},{error_pos_start,586},{error_pos_end,749},{errors,[{invalid_format,unexpected_time_format}]}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-15 00:45:15.169797 +0000\t1434329115\tleo_compact_fsm_worker:execute_1/4\t1036\t[{obj_container_path,\"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870\"},{error_pos_start,945},{error_pos_end,1108},{errors,[{invalid_format,unexpected_time_format}]}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-15 00:45:15.668450 +0000\t1434329115\tleo_compact_fsm_worker:execute_1/4\t1036\t[{obj_container_path,\"/var/lib/jenkins/jobs/Tests_for_aws-sdk_clients_with_LeoFS/workspace/leofs/package/leo_storage_0/avs/object/1.avs_63601547870\"},{error_pos_start,31501},{error_pos_end,31652},{errors,[{invalid_format,unexpected_time_format}]}]\r\n```\r\n\r\nAnd then, I've recognised S3 Client OR Leo Storage does not set correctly timestamp of an object when removing it as below:\r\n\r\n```erlang\r\n%% Error log during the compaction:\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:02.324330 +0000\t1434419342\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test7877/testFile s.org\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:23.739688 +0000\t1434419363\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:23.766323 +0000\t1434419363\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile.copy\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:29.770379 +0000\t1434419369\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile\\n2\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:29.774589 +0000\t1434419369\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:29.786912 +0000\t1434419369\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile.copy\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n[E]\tstorage_0@127.0.0.1\t2015-06-16 01:49:29.800619 +0000\t1434419369\tleo_object_storage_haystack:put_fun_2/3\t586\t[{key,<<\"test/testFile.single\\n1\">>},{del,1},{timestamp,0},{cause,\"Not set timestamp correctly\"}]\r\n```\r\n\r\n### Take a measure\r\n* Avoid inserting incorrect data into a AVS file and Set correct timestamp of an object as well as a metadata\r\n"}}