{"action":"created","issue":{"url":"https://api.github.com/repos/facebook/rocksdb/issues/498","labels_url":"https://api.github.com/repos/facebook/rocksdb/issues/498/labels{/name}","comments_url":"https://api.github.com/repos/facebook/rocksdb/issues/498/comments","events_url":"https://api.github.com/repos/facebook/rocksdb/issues/498/events","html_url":"https://github.com/facebook/rocksdb/issues/498","id":56934647,"number":498,"title":"What's the fastest way to export?","user":{"login":"vinniefalco","id":1503976,"avatar_url":"https://avatars.githubusercontent.com/u/1503976?v=3","gravatar_id":"","url":"https://api.github.com/users/vinniefalco","html_url":"https://github.com/vinniefalco","followers_url":"https://api.github.com/users/vinniefalco/followers","following_url":"https://api.github.com/users/vinniefalco/following{/other_user}","gists_url":"https://api.github.com/users/vinniefalco/gists{/gist_id}","starred_url":"https://api.github.com/users/vinniefalco/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vinniefalco/subscriptions","organizations_url":"https://api.github.com/users/vinniefalco/orgs","repos_url":"https://api.github.com/users/vinniefalco/repos","events_url":"https://api.github.com/users/vinniefalco/events{/privacy}","received_events_url":"https://api.github.com/users/vinniefalco/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-02-08T01:49:31Z","updated_at":"2015-02-08T12:37:27Z","closed_at":null,"body":"I'm working on a rocksdb to nudb importer and its taking seven minutes just to open the rocksdb database (784GB, 330k+ .sst files). On a non-raid spinning disk almost half an hour or more. The operating system shows that its opening each .sst file.\r\n\r\nI don't see any options to improve this, so I'm thinking about resorting to just iterating each .sst file myself. The code is a bit complex, are there any pointers on how to do this? Can I re-use some rocksdb code like plain_table_reader or something like that? I just need to iterate all the key/value pairs in the entire database in any order.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/facebook/rocksdb/issues/comments/73409081","html_url":"https://github.com/facebook/rocksdb/issues/498#issuecomment-73409081","issue_url":"https://api.github.com/repos/facebook/rocksdb/issues/498","id":73409081,"user":{"login":"ekg","id":145425,"avatar_url":"https://avatars.githubusercontent.com/u/145425?v=3","gravatar_id":"","url":"https://api.github.com/users/ekg","html_url":"https://github.com/ekg","followers_url":"https://api.github.com/users/ekg/followers","following_url":"https://api.github.com/users/ekg/following{/other_user}","gists_url":"https://api.github.com/users/ekg/gists{/gist_id}","starred_url":"https://api.github.com/users/ekg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ekg/subscriptions","organizations_url":"https://api.github.com/users/ekg/orgs","repos_url":"https://api.github.com/users/ekg/repos","events_url":"https://api.github.com/users/ekg/events{/privacy}","received_events_url":"https://api.github.com/users/ekg/received_events","type":"User","site_admin":false},"created_at":"2015-02-08T12:37:27Z","updated_at":"2015-02-08T12:37:27Z","body":"@vinniefalco Maybe your open would be much quicker if you compacted to a higher level. I think this would make more sense for workloads that are bulk load + many reads."}}