{"action":"created","issue":{"url":"https://api.github.com/repos/druid-io/druid/issues/1861","labels_url":"https://api.github.com/repos/druid-io/druid/issues/1861/labels{/name}","comments_url":"https://api.github.com/repos/druid-io/druid/issues/1861/comments","events_url":"https://api.github.com/repos/druid-io/druid/issues/1861/events","html_url":"https://github.com/druid-io/druid/pull/1861","id":113418518,"number":1861,"title":"insert-segment tool","user":{"login":"guobingkun","id":11789051,"avatar_url":"https://avatars.githubusercontent.com/u/11789051?v=3","gravatar_id":"","url":"https://api.github.com/users/guobingkun","html_url":"https://github.com/guobingkun","followers_url":"https://api.github.com/users/guobingkun/followers","following_url":"https://api.github.com/users/guobingkun/following{/other_user}","gists_url":"https://api.github.com/users/guobingkun/gists{/gist_id}","starred_url":"https://api.github.com/users/guobingkun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guobingkun/subscriptions","organizations_url":"https://api.github.com/users/guobingkun/orgs","repos_url":"https://api.github.com/users/guobingkun/repos","events_url":"https://api.github.com/users/guobingkun/events{/privacy}","received_events_url":"https://api.github.com/users/guobingkun/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-10-26T18:17:50Z","updated_at":"2015-10-26T21:30:58Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/druid-io/druid/pulls/1861","html_url":"https://github.com/druid-io/druid/pull/1861","diff_url":"https://github.com/druid-io/druid/pull/1861.diff","patch_url":"https://github.com/druid-io/druid/pull/1861.patch"},"body":"This tool can be used to load segments into Druid by inserting the segment's payload into metadata storage. People can use it to migrate segments to different deep storages.\r\n\r\nUsage example:\r\n```java -Ddruid.extensions.loadList=[\\\"mysql-metadata-storage\\\",\\\"druid-hdfs-storage\\\"] -cp $CLASSPATH io.druid.cli.Main tools insert-segment --workingDir hdfs://tmp/druid/localStorage/wikipedia/```\r\n\r\nSuppose under wikipedia, it looks like this,\r\n```\r\n├── 2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z\r\n│   └── 2015-10-21T22:07:57.074Z\r\n│       └── 0\r\n│           ├── descriptor.json\r\n│           └── index.zip\r\n├── 2013-09-01T00:00:00.000Z_2013-09-02T00:00:00.000Z\r\n│   └── 2015-10-21T22:07:57.074Z\r\n│       └── 0\r\n│           ├── descriptor.json\r\n│           └── index.zip\r\n├── 2013-09-02T00:00:00.000Z_2013-09-03T00:00:00.000Z\r\n│   └── 2015-10-21T22:07:57.074Z\r\n│       └── 0\r\n│           ├── descriptor.json\r\n│           └── index.zip\r\n└── 2013-09-03T00:00:00.000Z_2013-09-04T00:00:00.000Z\r\n    └── 2015-10-21T22:07:57.074Z\r\n        └── 0\r\n            ├── descriptor.json\r\n            └── index.zip\r\n```\r\n\r\nThen these 4 segments will be found and inserted into metadata storage, each segment's loadSpec will be updated with the location from where it was found.\r\n\r\nThis PR depends on https://github.com/druid-io/druid-api/pull/62\r\nDataSegmentFinder is used so that this tool can work on different deep storages. \r\nAn HDFS version is implemented in this PR."},"comment":{"url":"https://api.github.com/repos/druid-io/druid/issues/comments/151290449","html_url":"https://github.com/druid-io/druid/pull/1861#issuecomment-151290449","issue_url":"https://api.github.com/repos/druid-io/druid/issues/1861","id":151290449,"user":{"login":"drcrallen","id":8213081,"avatar_url":"https://avatars.githubusercontent.com/u/8213081?v=3","gravatar_id":"","url":"https://api.github.com/users/drcrallen","html_url":"https://github.com/drcrallen","followers_url":"https://api.github.com/users/drcrallen/followers","following_url":"https://api.github.com/users/drcrallen/following{/other_user}","gists_url":"https://api.github.com/users/drcrallen/gists{/gist_id}","starred_url":"https://api.github.com/users/drcrallen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drcrallen/subscriptions","organizations_url":"https://api.github.com/users/drcrallen/orgs","repos_url":"https://api.github.com/users/drcrallen/repos","events_url":"https://api.github.com/users/drcrallen/events{/privacy}","received_events_url":"https://api.github.com/users/drcrallen/received_events","type":"User","site_admin":false},"created_at":"2015-10-26T21:30:58Z","updated_at":"2015-10-26T21:30:58Z","body":"This tool does not interact with the interval locking right? That means you may end up with nasty race conditions if used on an active cluster."}}