{"action":"created","issue":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1490","labels_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1490/labels{/name}","comments_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1490/comments","events_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1490/events","html_url":"https://github.com/puppetlabs/puppetdb/pull/1490","id":90811748,"number":1490,"title":"(PDB-1631) master/add import and export endpoints","user":{"login":"ajroetker","id":3198124,"avatar_url":"https://avatars.githubusercontent.com/u/3198124?v=3","gravatar_id":"","url":"https://api.github.com/users/ajroetker","html_url":"https://github.com/ajroetker","followers_url":"https://api.github.com/users/ajroetker/followers","following_url":"https://api.github.com/users/ajroetker/following{/other_user}","gists_url":"https://api.github.com/users/ajroetker/gists{/gist_id}","starred_url":"https://api.github.com/users/ajroetker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajroetker/subscriptions","organizations_url":"https://api.github.com/users/ajroetker/orgs","repos_url":"https://api.github.com/users/ajroetker/repos","events_url":"https://api.github.com/users/ajroetker/events{/privacy}","received_events_url":"https://api.github.com/users/ajroetker/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/milestones/1","html_url":"https://github.com/puppetlabs/puppetdb/milestones/3.1.0%20(master)","labels_url":"https://api.github.com/repos/puppetlabs/puppetdb/milestones/1/labels","id":1187036,"number":1,"title":"3.1.0 (master)","description":"","creator":{"login":"kbarber","id":80595,"avatar_url":"https://avatars.githubusercontent.com/u/80595?v=3","gravatar_id":"","url":"https://api.github.com/users/kbarber","html_url":"https://github.com/kbarber","followers_url":"https://api.github.com/users/kbarber/followers","following_url":"https://api.github.com/users/kbarber/following{/other_user}","gists_url":"https://api.github.com/users/kbarber/gists{/gist_id}","starred_url":"https://api.github.com/users/kbarber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbarber/subscriptions","organizations_url":"https://api.github.com/users/kbarber/orgs","repos_url":"https://api.github.com/users/kbarber/repos","events_url":"https://api.github.com/users/kbarber/events{/privacy}","received_events_url":"https://api.github.com/users/kbarber/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":0,"state":"open","created_at":"2015-06-30T12:56:42Z","updated_at":"2015-06-30T16:45:13Z","due_on":null,"closed_at":null},"comments":9,"created_at":"2015-06-25T00:10:49Z","updated_at":"2015-06-30T21:01:21Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1490","html_url":"https://github.com/puppetlabs/puppetdb/pull/1490","diff_url":"https://github.com/puppetlabs/puppetdb/pull/1490.diff","patch_url":"https://github.com/puppetlabs/puppetdb/pull/1490.patch"},"body":"This PR adds a `puppetlabs.puppetdb.helpers` namespace.\r\nThe `helpers` namespace contains a trapperkeeper service which serves a handler of the new import/export endpoints.\r\n\r\nThis PR also creates a new subdirectory of the `vardir`, namely `backups`. This directory holds all the PuppetDB backup archives and is essentially the root directory of the `import` and `export` endpoints.\r\n\r\nThe new endpoints are served under the `/pdb/cli/v1` api root prefix.\r\n\r\nThe `export` endpoint will either stream you the backup tarball `GET`, or will trigger an export to the local `backups` directory `PUT`.\r\nThe streaming `GET` version takes an `archive` param which is useful for things like a browser, where the value supplied to the param will be the name of the file downloaded, for `curl` or Clojure calls to the endpoint, the param isn't necessary.\r\nThe local `PUT` version takes a param (`archive`) which is the name of the local backup tarball file you wish to make (in the future, this might be a timestamped default).\r\n\r\nThe import endpoints are `upload` and `import`.\r\nThe `upload` endpoint takes a multipart header (`curl ... -F \"archive=@foo_backup\"`) one of the params should be the `archive` file which you wish to upload to the PuppetDB backups directory.\r\nThe `import` endpoint takes a single `archive` param specifies which archive in the `backups` dir to import into PuppetDB, and executes the import.\r\n\r\nThe CLI tools (import/export) functionality has been moved to these endpoints and the tools themselves essentially just make calls to these endpoints. Import is an upload, followed by an import.\r\n\r\nTODO and followups for this PR:\r\nA `DELETE` endpoint for removing tarballs from the archive directory.\r\nMove the import/export logic out of the `cli` namespaces, maybe their own namespaces.\r\nAs to not be overreaching, this PR tried to allow the old import/export code to live next to the new code, as such there are a lot of `partial` floating around and anonymous functions be passed through the new/old code, this should be moved to just assume in-process command-submission and querying.\r\nMore validation around the compojure app.\r\nCounts of the exported objects in the final line.\r\nDon't reimport things that already exist.\r\nMore granular testing around the endpoints would be nice.\r\nAdd a benchmarking endpoint for data-loading, there is code duplication with the import tool there.\r\nDocumentation for the new endpoints.\r\n"},"comment":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/comments/117340780","html_url":"https://github.com/puppetlabs/puppetdb/pull/1490#issuecomment-117340780","issue_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1490","id":117340780,"user":{"login":"ajroetker","id":3198124,"avatar_url":"https://avatars.githubusercontent.com/u/3198124?v=3","gravatar_id":"","url":"https://api.github.com/users/ajroetker","html_url":"https://github.com/ajroetker","followers_url":"https://api.github.com/users/ajroetker/followers","following_url":"https://api.github.com/users/ajroetker/following{/other_user}","gists_url":"https://api.github.com/users/ajroetker/gists{/gist_id}","starred_url":"https://api.github.com/users/ajroetker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajroetker/subscriptions","organizations_url":"https://api.github.com/users/ajroetker/orgs","repos_url":"https://api.github.com/users/ajroetker/repos","events_url":"https://api.github.com/users/ajroetker/events{/privacy}","received_events_url":"https://api.github.com/users/ajroetker/received_events","type":"User","site_admin":false},"created_at":"2015-06-30T21:01:21Z","updated_at":"2015-06-30T21:01:21Z","body":"@pljenkinsro retest this please"}}