{"action":"created","issue":{"url":"https://api.github.com/repos/google/incremental-dom/issues/118","labels_url":"https://api.github.com/repos/google/incremental-dom/issues/118/labels{/name}","comments_url":"https://api.github.com/repos/google/incremental-dom/issues/118/comments","events_url":"https://api.github.com/repos/google/incremental-dom/issues/118/events","html_url":"https://github.com/google/incremental-dom/issues/118","id":103418527,"number":118,"title":"Create performance benchmarks","user":{"login":"sparhami","id":1410321,"avatar_url":"https://avatars.githubusercontent.com/u/1410321?v=3","gravatar_id":"","url":"https://api.github.com/users/sparhami","html_url":"https://github.com/sparhami","followers_url":"https://api.github.com/users/sparhami/followers","following_url":"https://api.github.com/users/sparhami/following{/other_user}","gists_url":"https://api.github.com/users/sparhami/gists{/gist_id}","starred_url":"https://api.github.com/users/sparhami/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sparhami/subscriptions","organizations_url":"https://api.github.com/users/sparhami/orgs","repos_url":"https://api.github.com/users/sparhami/repos","events_url":"https://api.github.com/users/sparhami/events{/privacy}","received_events_url":"https://api.github.com/users/sparhami/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-27T03:26:29Z","updated_at":"2015-08-27T19:06:23Z","closed_at":null,"body":"Currently, Incremental DOM does not have any good performance benchmarks to base changes on. Ideally they would be able to run as a part of the build process as well as standalone. In addition to testing for something like average time per pass, it should also check for the maximum time per pass to make sure GC is not kicking in.\r\n\r\nThis is fairly difficult as it is hard to model how the rest of an application generates garbage during updates. While the library itself may never generate enough garbage to cause more than an young gen GC, any allocation may end up ultimately causing old-gen GC which is hard to isolate and reproduce in a test, especially as heap sizes are likely to vary between environments. Since young gen GC can be very fast, it can be difficult to notice GC occurring with the amount of noise in a test.\r\n\r\nAdditionally, a spike does not necessarily indicate a GC occurred, but could happen due to the process being scheduled off which is likely to occur if running many iterations and hogging the CPU. This is not likely to occur in actual usage while updating web pages, but is something to watch out for when creating tests\r\n\r\nUltimately, any sort of environment we are likely to test on will have a huge amount of RAM compared to a smartphone in a developing country, which may have as little as 256MB and much of that taken by the OS."},"comment":{"url":"https://api.github.com/repos/google/incremental-dom/issues/comments/135525230","html_url":"https://github.com/google/incremental-dom/issues/118#issuecomment-135525230","issue_url":"https://api.github.com/repos/google/incremental-dom/issues/118","id":135525230,"user":{"login":"sparhami","id":1410321,"avatar_url":"https://avatars.githubusercontent.com/u/1410321?v=3","gravatar_id":"","url":"https://api.github.com/users/sparhami","html_url":"https://github.com/sparhami","followers_url":"https://api.github.com/users/sparhami/followers","following_url":"https://api.github.com/users/sparhami/following{/other_user}","gists_url":"https://api.github.com/users/sparhami/gists{/gist_id}","starred_url":"https://api.github.com/users/sparhami/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sparhami/subscriptions","organizations_url":"https://api.github.com/users/sparhami/orgs","repos_url":"https://api.github.com/users/sparhami/repos","events_url":"https://api.github.com/users/sparhami/events{/privacy}","received_events_url":"https://api.github.com/users/sparhami/received_events","type":"User","site_admin":false},"created_at":"2015-08-27T19:06:23Z","updated_at":"2015-08-27T19:06:23Z","body":"There may be some flags that can be used from https://code.google.com/p/v8/source/browse/branches/bleeding_edge/src/flag-definitions.h#522 to check for GCs perhaps by capturing and inspecting the trace output.\r\n\r\nSome flags from https://code.google.com/p/v8/source/browse/branches/bleeding_edge/src/flag-definitions.h#255 may be useful in disabling optimizations to allow for better measuring of performance on non-hot code paths while still capturing enough data to draw conclusions."}}