{"action":"created","comment":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/comments/47012651","id":47012651,"diff_hunk":"@@ -299,15 +300,44 @@\n           (app (assoc req :body-string payload))\n           (http/error-response (str \"Missing required parameter 'payload'\")))))))\n \n+(defn fail-when-payload-too-large\n+  \"Middlware that will return a 413 failure when the content-length of\n+  a POST is too large (more than half the max heap). Acts as a noop\n+  when that is not the case\"\n+  [app max-command-failure? max-command-size]\n+  (fn [req]\n+    (if (= :post (:request-method req))\n+      (let [length-in-bytes (request/content-length req)]\n+\n+        (if length-in-bytes\n+          (log/infof \"Processing command with a content-length of %s bytes\" length-in-bytes )\n+          (log/warn (str \"No content length found for POST. \"\n+                         \"POST bodies that are too large could cause memory-related failures.\")))\n+\n+        (if (and length-in-bytes\n+                 max-command-failure?\n+                 (> length-in-bytes max-command-size))\n+          (do\n+            (log/warnf \"content-length of command is %sMB and is larger than the maximum allowed command size of %sMB\"","path":"src/puppetlabs/puppetdb/middleware.clj","position":32,"original_position":32,"commit_id":"bf0ca8988b21340908648aaf8399661bc418539f","original_commit_id":"181357db8212e6bdb4ea4b7017fc8715d36dd3eb","user":{"login":"kbarber","id":80595,"avatar_url":"https://avatars.githubusercontent.com/u/80595?v=3","gravatar_id":"","url":"https://api.github.com/users/kbarber","html_url":"https://github.com/kbarber","followers_url":"https://api.github.com/users/kbarber/followers","following_url":"https://api.github.com/users/kbarber/following{/other_user}","gists_url":"https://api.github.com/users/kbarber/gists{/gist_id}","starred_url":"https://api.github.com/users/kbarber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbarber/subscriptions","organizations_url":"https://api.github.com/users/kbarber/orgs","repos_url":"https://api.github.com/users/kbarber/repos","events_url":"https://api.github.com/users/kbarber/events{/privacy}","received_events_url":"https://api.github.com/users/kbarber/received_events","type":"User","site_admin":false},"body":"This ends up looking like:\r\n\r\n```\r\n2015-12-08 20:54:29,940 WARN  [qtp752077813-86 - /pdb/cmd/v1?checksum=95f7fbca7440767b134a1ddb748db27552e0152d] [p.p.middleware] content-length of command is 6.180831MB and is larger than the maximum allowed command size of 1.0MB\r\n```\r\n\r\nI'd say, space between the float and 'MB', and maybe consider rounding that float to 2-3 digits. It just looks crazy with six digits like this, might as well be a byte :-).","created_at":"2015-12-08T20:59:21Z","updated_at":"2015-12-08T20:59:21Z","html_url":"https://github.com/puppetlabs/puppetdb/pull/1763#discussion_r47012651","pull_request_url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763","_links":{"self":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/comments/47012651"},"html":{"href":"https://github.com/puppetlabs/puppetdb/pull/1763#discussion_r47012651"},"pull_request":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763"}}},"pull_request":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763","id":51715218,"html_url":"https://github.com/puppetlabs/puppetdb/pull/1763","diff_url":"https://github.com/puppetlabs/puppetdb/pull/1763.diff","patch_url":"https://github.com/puppetlabs/puppetdb/pull/1763.patch","issue_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1763","number":1763,"state":"open","locked":false,"title":"(PDB-156) Fail gracefully when commands are too large","user":{"login":"senior","id":215910,"avatar_url":"https://avatars.githubusercontent.com/u/215910?v=3","gravatar_id":"","url":"https://api.github.com/users/senior","html_url":"https://github.com/senior","followers_url":"https://api.github.com/users/senior/followers","following_url":"https://api.github.com/users/senior/following{/other_user}","gists_url":"https://api.github.com/users/senior/gists{/gist_id}","starred_url":"https://api.github.com/users/senior/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senior/subscriptions","organizations_url":"https://api.github.com/users/senior/orgs","repos_url":"https://api.github.com/users/senior/repos","events_url":"https://api.github.com/users/senior/events{/privacy}","received_events_url":"https://api.github.com/users/senior/received_events","type":"User","site_admin":false},"body":"This commit uses the content-length header to determine the size of the\r\nincoming request. If that POST request is too large for the system to\r\nprocess (currently set at 1/205th of the max heap size) it will return\r\nclose the input stream and return a 412 status code.","created_at":"2015-11-24T20:50:55Z","updated_at":"2015-12-08T20:59:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":"5c1a3173335c92370ad74d7ea4819ce74351947f","assignee":null,"milestone":{"url":"https://api.github.com/repos/puppetlabs/puppetdb/milestones/16","html_url":"https://github.com/puppetlabs/puppetdb/milestones/3.2.3%20(stable)","labels_url":"https://api.github.com/repos/puppetlabs/puppetdb/milestones/16/labels","id":1440674,"number":16,"title":"3.2.3 (stable)","description":"","creator":{"login":"kbarber","id":80595,"avatar_url":"https://avatars.githubusercontent.com/u/80595?v=3","gravatar_id":"","url":"https://api.github.com/users/kbarber","html_url":"https://github.com/kbarber","followers_url":"https://api.github.com/users/kbarber/followers","following_url":"https://api.github.com/users/kbarber/following{/other_user}","gists_url":"https://api.github.com/users/kbarber/gists{/gist_id}","starred_url":"https://api.github.com/users/kbarber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbarber/subscriptions","organizations_url":"https://api.github.com/users/kbarber/orgs","repos_url":"https://api.github.com/users/kbarber/repos","events_url":"https://api.github.com/users/kbarber/events{/privacy}","received_events_url":"https://api.github.com/users/kbarber/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":2,"state":"open","created_at":"2015-12-02T19:58:53Z","updated_at":"2015-12-04T16:15:17Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763/commits","review_comments_url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763/comments","review_comment_url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/comments{/number}","comments_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1763/comments","statuses_url":"https://api.github.com/repos/puppetlabs/puppetdb/statuses/bf0ca8988b21340908648aaf8399661bc418539f","head":{"label":"senior:ticket/stable/pdb-156-refuse-large-payloads","ref":"ticket/stable/pdb-156-refuse-large-payloads","sha":"bf0ca8988b21340908648aaf8399661bc418539f","user":{"login":"senior","id":215910,"avatar_url":"https://avatars.githubusercontent.com/u/215910?v=3","gravatar_id":"","url":"https://api.github.com/users/senior","html_url":"https://github.com/senior","followers_url":"https://api.github.com/users/senior/followers","following_url":"https://api.github.com/users/senior/following{/other_user}","gists_url":"https://api.github.com/users/senior/gists{/gist_id}","starred_url":"https://api.github.com/users/senior/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senior/subscriptions","organizations_url":"https://api.github.com/users/senior/orgs","repos_url":"https://api.github.com/users/senior/repos","events_url":"https://api.github.com/users/senior/events{/privacy}","received_events_url":"https://api.github.com/users/senior/received_events","type":"User","site_admin":false},"repo":{"id":13152306,"name":"puppetdb","full_name":"senior/puppetdb","owner":{"login":"senior","id":215910,"avatar_url":"https://avatars.githubusercontent.com/u/215910?v=3","gravatar_id":"","url":"https://api.github.com/users/senior","html_url":"https://github.com/senior","followers_url":"https://api.github.com/users/senior/followers","following_url":"https://api.github.com/users/senior/following{/other_user}","gists_url":"https://api.github.com/users/senior/gists{/gist_id}","starred_url":"https://api.github.com/users/senior/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senior/subscriptions","organizations_url":"https://api.github.com/users/senior/orgs","repos_url":"https://api.github.com/users/senior/repos","events_url":"https://api.github.com/users/senior/events{/privacy}","received_events_url":"https://api.github.com/users/senior/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/senior/puppetdb","description":"Centralized Puppet Storage","fork":true,"url":"https://api.github.com/repos/senior/puppetdb","forks_url":"https://api.github.com/repos/senior/puppetdb/forks","keys_url":"https://api.github.com/repos/senior/puppetdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/senior/puppetdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/senior/puppetdb/teams","hooks_url":"https://api.github.com/repos/senior/puppetdb/hooks","issue_events_url":"https://api.github.com/repos/senior/puppetdb/issues/events{/number}","events_url":"https://api.github.com/repos/senior/puppetdb/events","assignees_url":"https://api.github.com/repos/senior/puppetdb/assignees{/user}","branches_url":"https://api.github.com/repos/senior/puppetdb/branches{/branch}","tags_url":"https://api.github.com/repos/senior/puppetdb/tags","blobs_url":"https://api.github.com/repos/senior/puppetdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/senior/puppetdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/senior/puppetdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/senior/puppetdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/senior/puppetdb/statuses/{sha}","languages_url":"https://api.github.com/repos/senior/puppetdb/languages","stargazers_url":"https://api.github.com/repos/senior/puppetdb/stargazers","contributors_url":"https://api.github.com/repos/senior/puppetdb/contributors","subscribers_url":"https://api.github.com/repos/senior/puppetdb/subscribers","subscription_url":"https://api.github.com/repos/senior/puppetdb/subscription","commits_url":"https://api.github.com/repos/senior/puppetdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/senior/puppetdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/senior/puppetdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/senior/puppetdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/senior/puppetdb/contents/{+path}","compare_url":"https://api.github.com/repos/senior/puppetdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/senior/puppetdb/merges","archive_url":"https://api.github.com/repos/senior/puppetdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/senior/puppetdb/downloads","issues_url":"https://api.github.com/repos/senior/puppetdb/issues{/number}","pulls_url":"https://api.github.com/repos/senior/puppetdb/pulls{/number}","milestones_url":"https://api.github.com/repos/senior/puppetdb/milestones{/number}","notifications_url":"https://api.github.com/repos/senior/puppetdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/senior/puppetdb/labels{/name}","releases_url":"https://api.github.com/repos/senior/puppetdb/releases{/id}","created_at":"2013-09-27T14:18:50Z","updated_at":"2014-06-21T06:15:55Z","pushed_at":"2015-12-08T20:58:53Z","git_url":"git://github.com/senior/puppetdb.git","ssh_url":"git@github.com:senior/puppetdb.git","clone_url":"https://github.com/senior/puppetdb.git","svn_url":"https://github.com/senior/puppetdb","homepage":"http://docs.puppetlabs.com/puppetdb","size":15174,"stargazers_count":0,"watchers_count":0,"language":"Clojure","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"puppetlabs:stable","ref":"stable","sha":"1bebc4f8e23595573339cb011d391399c9f4cdf7","user":{"login":"puppetlabs","id":234268,"avatar_url":"https://avatars.githubusercontent.com/u/234268?v=3","gravatar_id":"","url":"https://api.github.com/users/puppetlabs","html_url":"https://github.com/puppetlabs","followers_url":"https://api.github.com/users/puppetlabs/followers","following_url":"https://api.github.com/users/puppetlabs/following{/other_user}","gists_url":"https://api.github.com/users/puppetlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/puppetlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/puppetlabs/subscriptions","organizations_url":"https://api.github.com/users/puppetlabs/orgs","repos_url":"https://api.github.com/users/puppetlabs/repos","events_url":"https://api.github.com/users/puppetlabs/events{/privacy}","received_events_url":"https://api.github.com/users/puppetlabs/received_events","type":"Organization","site_admin":false},"repo":{"id":2193032,"name":"puppetdb","full_name":"puppetlabs/puppetdb","owner":{"login":"puppetlabs","id":234268,"avatar_url":"https://avatars.githubusercontent.com/u/234268?v=3","gravatar_id":"","url":"https://api.github.com/users/puppetlabs","html_url":"https://github.com/puppetlabs","followers_url":"https://api.github.com/users/puppetlabs/followers","following_url":"https://api.github.com/users/puppetlabs/following{/other_user}","gists_url":"https://api.github.com/users/puppetlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/puppetlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/puppetlabs/subscriptions","organizations_url":"https://api.github.com/users/puppetlabs/orgs","repos_url":"https://api.github.com/users/puppetlabs/repos","events_url":"https://api.github.com/users/puppetlabs/events{/privacy}","received_events_url":"https://api.github.com/users/puppetlabs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/puppetlabs/puppetdb","description":"Centralized Puppet Storage","fork":false,"url":"https://api.github.com/repos/puppetlabs/puppetdb","forks_url":"https://api.github.com/repos/puppetlabs/puppetdb/forks","keys_url":"https://api.github.com/repos/puppetlabs/puppetdb/keys{/key_id}","collaborators_url":"https://api.github.com/repos/puppetlabs/puppetdb/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/puppetlabs/puppetdb/teams","hooks_url":"https://api.github.com/repos/puppetlabs/puppetdb/hooks","issue_events_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/events{/number}","events_url":"https://api.github.com/repos/puppetlabs/puppetdb/events","assignees_url":"https://api.github.com/repos/puppetlabs/puppetdb/assignees{/user}","branches_url":"https://api.github.com/repos/puppetlabs/puppetdb/branches{/branch}","tags_url":"https://api.github.com/repos/puppetlabs/puppetdb/tags","blobs_url":"https://api.github.com/repos/puppetlabs/puppetdb/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/puppetlabs/puppetdb/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/puppetlabs/puppetdb/git/refs{/sha}","trees_url":"https://api.github.com/repos/puppetlabs/puppetdb/git/trees{/sha}","statuses_url":"https://api.github.com/repos/puppetlabs/puppetdb/statuses/{sha}","languages_url":"https://api.github.com/repos/puppetlabs/puppetdb/languages","stargazers_url":"https://api.github.com/repos/puppetlabs/puppetdb/stargazers","contributors_url":"https://api.github.com/repos/puppetlabs/puppetdb/contributors","subscribers_url":"https://api.github.com/repos/puppetlabs/puppetdb/subscribers","subscription_url":"https://api.github.com/repos/puppetlabs/puppetdb/subscription","commits_url":"https://api.github.com/repos/puppetlabs/puppetdb/commits{/sha}","git_commits_url":"https://api.github.com/repos/puppetlabs/puppetdb/git/commits{/sha}","comments_url":"https://api.github.com/repos/puppetlabs/puppetdb/comments{/number}","issue_comment_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues/comments{/number}","contents_url":"https://api.github.com/repos/puppetlabs/puppetdb/contents/{+path}","compare_url":"https://api.github.com/repos/puppetlabs/puppetdb/compare/{base}...{head}","merges_url":"https://api.github.com/repos/puppetlabs/puppetdb/merges","archive_url":"https://api.github.com/repos/puppetlabs/puppetdb/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/puppetlabs/puppetdb/downloads","issues_url":"https://api.github.com/repos/puppetlabs/puppetdb/issues{/number}","pulls_url":"https://api.github.com/repos/puppetlabs/puppetdb/pulls{/number}","milestones_url":"https://api.github.com/repos/puppetlabs/puppetdb/milestones{/number}","notifications_url":"https://api.github.com/repos/puppetlabs/puppetdb/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/puppetlabs/puppetdb/labels{/name}","releases_url":"https://api.github.com/repos/puppetlabs/puppetdb/releases{/id}","created_at":"2011-08-11T18:54:46Z","updated_at":"2015-12-04T14:50:21Z","pushed_at":"2015-12-08T20:58:53Z","git_url":"git://github.com/puppetlabs/puppetdb.git","ssh_url":"git@github.com:puppetlabs/puppetdb.git","clone_url":"https://github.com/puppetlabs/puppetdb.git","svn_url":"https://github.com/puppetlabs/puppetdb","homepage":"http://docs.puppetlabs.com/puppetdb","size":18454,"stargazers_count":178,"watchers_count":178,"language":"Clojure","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":140,"mirror_url":null,"open_issues_count":6,"forks":140,"open_issues":6,"watchers":178,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763"},"html":{"href":"https://github.com/puppetlabs/puppetdb/pull/1763"},"issue":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1763"},"comments":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/issues/1763/comments"},"review_comments":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763/comments"},"review_comment":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/pulls/1763/commits"},"statuses":{"href":"https://api.github.com/repos/puppetlabs/puppetdb/statuses/bf0ca8988b21340908648aaf8399661bc418539f"}}}}