{"action":"opened","issue":{"url":"https://api.github.com/repos/tbreloff/Plots.jl/issues/38","labels_url":"https://api.github.com/repos/tbreloff/Plots.jl/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/tbreloff/Plots.jl/issues/38/comments","events_url":"https://api.github.com/repos/tbreloff/Plots.jl/issues/38/events","html_url":"https://github.com/tbreloff/Plots.jl/issues/38","id":108912595,"number":38,"title":"Testing","user":{"login":"tbreloff","id":933338,"avatar_url":"https://avatars.githubusercontent.com/u/933338?v=3","gravatar_id":"","url":"https://api.github.com/users/tbreloff","html_url":"https://github.com/tbreloff","followers_url":"https://api.github.com/users/tbreloff/followers","following_url":"https://api.github.com/users/tbreloff/following{/other_user}","gists_url":"https://api.github.com/users/tbreloff/gists{/gist_id}","starred_url":"https://api.github.com/users/tbreloff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tbreloff/subscriptions","organizations_url":"https://api.github.com/users/tbreloff/orgs","repos_url":"https://api.github.com/users/tbreloff/repos","events_url":"https://api.github.com/users/tbreloff/events{/privacy}","received_events_url":"https://api.github.com/users/tbreloff/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-29T16:57:46Z","updated_at":"2015-09-29T16:57:46Z","closed_at":null,"body":"Need framework to do regression testing automatically for changes in image output.  Likely I'll want to use the new test @timholy is adding here: https://github.com/timholy/Images.jl/pull/367\r\n\r\nHow should it be structured?  I'm thinking maybe a `test/ref_img` directory, with a subdirectory for each backend, and plots ordered according to the examples/test plots.  Then the automated tests could write out the newly generated png files to a temporary directory, then compare to the images in `test/ref_img`.\r\n\r\nIf images compare, great.  If not... I think ideally we could bring up a gui of some sort, which can show how the images differ, and give a nice dialog to optionally save the newly generated image as the new \"reference image\".\r\n\r\nThoughts on this process?  Should this be part of travis tests, or only something to be run locally by us?"}}