{"action":"created","issue":{"url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/3","labels_url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/3/comments","events_url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/3/events","html_url":"https://github.com/voxmedia/image_compression_experiments/issues/3","id":105478330,"number":3,"title":"Consider comparing only one variable at a time","user":{"login":"pornel","id":72159,"avatar_url":"https://avatars.githubusercontent.com/u/72159?v=3","gravatar_id":"","url":"https://api.github.com/users/pornel","html_url":"https://github.com/pornel","followers_url":"https://api.github.com/users/pornel/followers","following_url":"https://api.github.com/users/pornel/following{/other_user}","gists_url":"https://api.github.com/users/pornel/gists{/gist_id}","starred_url":"https://api.github.com/users/pornel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pornel/subscriptions","organizations_url":"https://api.github.com/users/pornel/orgs","repos_url":"https://api.github.com/users/pornel/repos","events_url":"https://api.github.com/users/pornel/events{/privacy}","received_events_url":"https://api.github.com/users/pornel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-09-08T22:01:12Z","updated_at":"2015-09-09T14:09:15Z","closed_at":null,"body":"Image file sizes often grow exponentially(ish) with quality, so even barely noticeable differences in quality can give you wildly different file sizes and misleading results (more on that https://pornel.net/faircomparison).\r\n\r\nWhen comparing codecs usually the goal is to find one with best quality/filesize ratio — even if you plan to convert images at a fixed quality setting, the aim is still to get best \"bang for the buck\" (i.e if the fixed quality setting turns out to be too high for an image, you still want to minimize filesize cost of that).\r\n\r\nHowever, when in test results you have a result where the file is bigger, but quality is also better, it's impossible to tell whether quality/filesize ratio is better or worse, because that relationship is complex and non-linear.\r\n\r\nThe best way is to compare only one variable (keeping the other the same), and ensure results are in the same order of magnitude to avoid one exceptionally large value from deciding fate of the whole set (e.g. imagine comparing savings on images in KB range, and one in multi-MB range. No matter how good or bad the small images are, even small changes to the multi-MB one will be bigger than that, shadowing all other results).\r\n\r\n\r\nFor example if you're planning to convert images to a fixed quality setting, adjust that setting so that the sum of file sizes for all codecs is the same, and then compare quality.\r\n\r\nOr convert images to the same, precisely measured quality, and then file sizes will give you indication how much on average you can save.\r\n"},"comment":{"url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/comments/138921193","html_url":"https://github.com/voxmedia/image_compression_experiments/issues/3#issuecomment-138921193","issue_url":"https://api.github.com/repos/voxmedia/image_compression_experiments/issues/3","id":138921193,"user":{"login":"okor","id":142910,"avatar_url":"https://avatars.githubusercontent.com/u/142910?v=3","gravatar_id":"","url":"https://api.github.com/users/okor","html_url":"https://github.com/okor","followers_url":"https://api.github.com/users/okor/followers","following_url":"https://api.github.com/users/okor/following{/other_user}","gists_url":"https://api.github.com/users/okor/gists{/gist_id}","starred_url":"https://api.github.com/users/okor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/okor/subscriptions","organizations_url":"https://api.github.com/users/okor/orgs","repos_url":"https://api.github.com/users/okor/repos","events_url":"https://api.github.com/users/okor/events{/privacy}","received_events_url":"https://api.github.com/users/okor/received_events","type":"User","site_admin":false},"created_at":"2015-09-09T14:09:15Z","updated_at":"2015-09-09T14:09:15Z","body":"Thanks for the feedback!\r\n\r\nIt's possible I may have reinvented a wheel here. I'm curious if you know of some tool that might already be out there (open source) which implements the kind of quality/filesize ratio experiment that I am trying to conduct here ... but the implementation is closer to the spec you have described. Know of anything?\r\n\r\nIdeally, we would have an image processing server which was a bit smarter so this experiment would be less necessary. Rather than specify a \"quality\" ... specify a DSSIM/SSIM tolerance. So, \"make this image as small as possible while within this DSSIM tolerance\", would be the goal. But to the best of my knowledge (please correct me if I am wrong) encoders don't have this kind of capability built in, so the parent application would need to use some type of iterative process for each image ... which would be very slow and not well suited for something that should generated a resized and encode image quickly in an HTTP request cycle. Something akin to Tobias Baldauf's adept script. I have however started dreaming up an application design that does just what outlined here but to get around the speed issue, return a \"safe\" image quickly but set the cache headers to immediately expire. Then in the background, do all of the hard iterative work. Then once the \"best\" image is determined, respond to all later requests with that \"best\" image along with long lived cache headers. That way your CDN or Varnish or whatever can hold onto that image forever."}}