{"action":"created","issue":{"url":"https://api.github.com/repos/photoshell/photoshell/issues/96","labels_url":"https://api.github.com/repos/photoshell/photoshell/issues/96/labels{/name}","comments_url":"https://api.github.com/repos/photoshell/photoshell/issues/96/comments","events_url":"https://api.github.com/repos/photoshell/photoshell/issues/96/events","html_url":"https://github.com/photoshell/photoshell/issues/96","id":54665366,"number":96,"title":"Hashing optimization","user":{"login":"SamWhited","id":512573,"avatar_url":"https://avatars.githubusercontent.com/u/512573?v=3","gravatar_id":"","url":"https://api.github.com/users/SamWhited","html_url":"https://github.com/SamWhited","followers_url":"https://api.github.com/users/SamWhited/followers","following_url":"https://api.github.com/users/SamWhited/following{/other_user}","gists_url":"https://api.github.com/users/SamWhited/gists{/gist_id}","starred_url":"https://api.github.com/users/SamWhited/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SamWhited/subscriptions","organizations_url":"https://api.github.com/users/SamWhited/orgs","repos_url":"https://api.github.com/users/SamWhited/repos","events_url":"https://api.github.com/users/SamWhited/events{/privacy}","received_events_url":"https://api.github.com/users/SamWhited/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/photoshell/photoshell/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-01-17T15:51:10Z","updated_at":"2015-01-17T17:50:59Z","closed_at":null,"body":"Working on returning metadata in rawphoto. One thing it would return is the hash which we'll use to identify the image later. That got me thinking about how we could speed up the hashing on slow machines.\r\n\r\nDo we really need the hash for the entire file?\r\n\r\nWhat if we checked if certain identifying metadata fields were present, and then just hashed the metadata and used that as the identifier? If those fields aren't present, only then do we hash the entire file.\r\n\r\nThe actual raw file should never change after it comes out of the camera, so I think this would work. Fields that we could look for include a combination of sequence number, unique identifier, datetime, etc. If we have a few mostly-unique items (sequence number may repeat, but pair it with datetime and we're good; multiple photos may be taken at the exact tame time, but add the camera's serial number and se can see that they were different, etc.), we just hash the metadata blob. Otherwise, we hash the entire file.\r\n\r\nThis would effectively change our storage method from hash's to UUIDs (which are still hashes, but we don't know of what necessarily).\r\n\r\nThoughts?"},"comment":{"url":"https://api.github.com/repos/photoshell/photoshell/issues/comments/70376459","html_url":"https://github.com/photoshell/photoshell/issues/96#issuecomment-70376459","issue_url":"https://api.github.com/repos/photoshell/photoshell/issues/96","id":70376459,"user":{"login":"campaul","id":552768,"avatar_url":"https://avatars.githubusercontent.com/u/552768?v=3","gravatar_id":"","url":"https://api.github.com/users/campaul","html_url":"https://github.com/campaul","followers_url":"https://api.github.com/users/campaul/followers","following_url":"https://api.github.com/users/campaul/following{/other_user}","gists_url":"https://api.github.com/users/campaul/gists{/gist_id}","starred_url":"https://api.github.com/users/campaul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/campaul/subscriptions","organizations_url":"https://api.github.com/users/campaul/orgs","repos_url":"https://api.github.com/users/campaul/repos","events_url":"https://api.github.com/users/campaul/events{/privacy}","received_events_url":"https://api.github.com/users/campaul/received_events","type":"User","site_admin":false},"created_at":"2015-01-17T17:50:59Z","updated_at":"2015-01-17T17:50:59Z","body":"Based on what I was seeing last night the bottleneck wasn't the hashing function, it was loading the file over USB2. Since we have to do that anyway to develop the raw, any time lost hashing is gained back by dcraw getting to load the now cached file."}}