{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4886","labels_url":"https://api.github.com/repos/apache/spark/issues/4886/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4886/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4886/events","html_url":"https://github.com/apache/spark/pull/4886","id":59759362,"number":4886,"title":"[SPARK-6156][CORE]Not cache in memory again while first put memory_disk block in memory failed.","user":{"login":"suyanNone","id":9818852,"avatar_url":"https://avatars.githubusercontent.com/u/9818852?v=3","gravatar_id":"","url":"https://api.github.com/users/suyanNone","html_url":"https://github.com/suyanNone","followers_url":"https://api.github.com/users/suyanNone/followers","following_url":"https://api.github.com/users/suyanNone/following{/other_user}","gists_url":"https://api.github.com/users/suyanNone/gists{/gist_id}","starred_url":"https://api.github.com/users/suyanNone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/suyanNone/subscriptions","organizations_url":"https://api.github.com/users/suyanNone/orgs","repos_url":"https://api.github.com/users/suyanNone/repos","events_url":"https://api.github.com/users/suyanNone/events{/privacy}","received_events_url":"https://api.github.com/users/suyanNone/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-03-04T07:15:55Z","updated_at":"2015-04-15T12:50:06Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4886","html_url":"https://github.com/apache/spark/pull/4886","diff_url":"https://github.com/apache/spark/pull/4886.diff","patch_url":"https://github.com/apache/spark/pull/4886.patch"},"body":"Current code:\r\nNow we want to cache a Memory_and_disk level block\r\n1. Try to put in memory and unroll unsuccessful. then reserved unroll memory because we got a iterator from an unroll Array \r\n2. Then put into disk.\r\n3. Get value from get(blockId), and iterator from that value.\r\n\r\nAnd there have some doubts:\r\nfor process 3:\r\n* get value use `blockManager.get(blockId)`\r\nin `get(blockId)` method, it will first getLocal() and then getRemote() in logical, I think is more suitable for use getLocal(blockId) to instead get(blockId), I dont know why we call get(blockId), have other considerations?\r\n\r\n* and in doGetLocal method, if it get value success from disk, it will try to put in memory again... I think, that a redundant...  we just try to put in memory and failed to unroll....\r\n\r\nmay this pull request is useless, because may have some reasons to do that?"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/93378547","html_url":"https://github.com/apache/spark/pull/4886#issuecomment-93378547","issue_url":"https://api.github.com/repos/apache/spark/issues/4886","id":93378547,"user":{"login":"srowen","id":822522,"avatar_url":"https://avatars.githubusercontent.com/u/822522?v=3","gravatar_id":"","url":"https://api.github.com/users/srowen","html_url":"https://github.com/srowen","followers_url":"https://api.github.com/users/srowen/followers","following_url":"https://api.github.com/users/srowen/following{/other_user}","gists_url":"https://api.github.com/users/srowen/gists{/gist_id}","starred_url":"https://api.github.com/users/srowen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/srowen/subscriptions","organizations_url":"https://api.github.com/users/srowen/orgs","repos_url":"https://api.github.com/users/srowen/repos","events_url":"https://api.github.com/users/srowen/events{/privacy}","received_events_url":"https://api.github.com/users/srowen/received_events","type":"User","site_admin":false},"created_at":"2015-04-15T12:50:06Z","updated_at":"2015-04-15T12:50:06Z","body":"@suyanNone from the description I'm still not sure I understand why this is needed. It seems reasonable to try again to put the data into memory, later, if it is intended to be stored in memory. The call from `CacheManager` now always skips putting data into memory, since the modified `if` condition becomes always true because `isDiskToMemory == !level.isMemory`. I think this is not the right change, based on what I understand."}}